* feat: limit the number of subscriptions

Closing #729

* fix nit

* Update core/src/server/helpers.rs

* add integration tests + some fixes so it works

* cargo fmt

* fix doc links

* Unsubscribe calls should avoid subscription limits

Point to Tokio 1.16 (we use a method from it), and a little special treatment for unsubscribe methods

* No resource limiting for Unsubscribe calls

* Test that we can still unsubscribe after hitting a limit

* Fix a comment typo

Co-authored-by: Alexandru Vasile <[email protected]>

* Update core/src/server/rpc_module.rs

* Update core/src/server/rpc_module.rs

Co-authored-by: James Wilson <[email protected]>
Co-authored-by: Alexandru Vasile <[email protected]>

* fix(rpc module): fail subscription with bad params

* draft; show my point

* fix tests

* fix build

* add tests for proc macros too

* add tests for bad params in proc macros

* fix nits

* commit all files

* add ugly fix for proc macro code

* add more user friendly API

* make SubscriptionSink::close take mut self

* fix grumbles

* show james some code

* Update core/src/server/rpc_module.rs

Co-authored-by: James Wilson <[email protected]>

* remove needless clone

* fix build

* client fix docs + error type

* simplify code: merge connect reset and unsubscribe close reason

* remove unknown close reason

* refactor: remove Error::SubscriptionClosed

* add some nice APIs to ErrorObjectOwned

* unify api

* address grumbles

* remove redundant methods for close and reject

* proc macro: compile err when subscription -> Result

* rpc module: fix test subscription test

* Update core/src/server/rpc_module.rs

Co-authored-by: James Wilson <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: James Wilson <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: James Wilson <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: James Wilson <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: James Wilson <[email protected]>

* Update proc-macros/src/lib.rs

Co-authored-by: James Wilson <[email protected]>

* address grumbles

* remove faulty comment

* Update core/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* fix: don't send `RPC Call failed: error`.

* remove debug assert

Co-authored-by: James Wilson <[email protected]>
Co-authored-by: David <[email protected]>

These logs are most likely related to that the client terminated the connection
and they come with significant overhead.

* fix(ws server): close all subscriptions

* fix test

* better comment and longer period

* revert unintentional change

* refactor: get rid off `Error::Request` variant

* fix nit

* to_owned -> to_call_error

* fix: max_request_body_size setting should not override max_response_size

* chore: apply review comment

* wip wip wip

Use tokio::sync::Notify to signal to the server when a subscriber has gone away without calling unsubscribe

* Cleanup

* Fmt

* More cleanup more TODOs

* fmt

* Address a few todos

* Update core/src/server/rpc_module.rs

Co-authored-by: Niklas Adolfsson <[email protected]>

* Update ws-server/src/server.rs

Co-authored-by: Niklas Adolfsson <[email protected]>

* Fix bad merge

* Test

* fmt

* fix test

* Finish test

* Cleanup
Add a second subscription to serverless test

* Update tests/tests/integration_tests.rs

Co-authored-by: Niklas Adolfsson <[email protected]>

* simplify test

* Review feedback: avoid allocations

* cleanup

* Remove async-channel

* remove async-channel deps

Co-authored-by: Niklas Adolfsson <[email protected]>

* fix(rpc module): unsubscribe ethereum pubsub spec

* update docs

* fix clippy

* fix tests

* refactor(ws server): set_id_provider `Box<dyn>`

* address grumbles: thanks James

* relax trait bounds with ?Sized

* Update ws-server/src/tests.rs

* add docs

* feat(rpc module): add_stream to subscription sink

* fix some nits

* unify parameters to rpc methods

* Update core/src/server/rpc_module.rs

* Update tests/tests/integration_tests.rs

Co-authored-by: David <[email protected]>

* address grumbles

* fix subscription tests

* new type for `SubscriptionCallback` and glue code

* remove unsed code

* remove todo

* add missing feature tokio/macros

* make `add_stream` cancel-safe

* rename add_stream and return status

* fix nits

* rename stream API -> streamify

* Update core/src/server/rpc_module.rs

* provide proper close reason

* spelling

* consume_and_streamify + docs

* fmt

* rename API pipe_from_stream

* improve logging; indicate which subscription method that failed

Co-authored-by: David <[email protected]>

Implies MSRV 1.56.x

* PoC

* introduce IdProvider trait

* revert Cow stuff

* Update ws-server/src/server.rs

* fix tests

* cargo fmt

* use 'static lifetime in SubscriptionId

* fix tests

* fmt

* make tests compile again

* fix tests

* Fix tests

* Move IdProvider impls to own module

* move `sub-id gen` types to `core`

* fix doc links

* make rand non-optional dep

* feature gate: id provider

Co-authored-by: David Palm <[email protected]>

* extract ws client to async client

* bring back jsonrpsee-ws-client crate

* new crate core client

* add missing file

* jsonrpsee crate: add core client

* rexport core client

* more re-exports

* downgrade trait bounds

* update version

* fix nits

* send close reason

* Update types/src/traits.rs

* move 'TEN_MB_CONST' to core

* Update client/core-client/Cargo.toml

Co-authored-by: David <[email protected]>

* Update client/core-client/src/lib.rs

Co-authored-by: David <[email protected]>

* cargo fmt

* Update client/ws-client/src/lib.rs

Co-authored-by: David <[email protected]>

* Update client/ws-client/src/lib.rs

Co-authored-by: David <[email protected]>

* move `async-client` core

The `async-client` is hidden behind a new feature flag `async-client`
because it brings in additional dependecies such as tokio rt.

* fix docs

* add example how to use "core client"

* fix build

* Update http-server/Cargo.toml

* Update client/transport/Cargo.toml

Co-authored-by: David <[email protected]>

* Nuke V2

* fmt

* Formatting and imports

* Updated benches

* Fix doc comment link

* Brace imports in ws-server

* Reworking imports

* std first

* fmt

* std on top

* Update to match changed line numbers

* Rename jsonrpsee_utils -> jsonrpsee_core

* Migrating things types -> core

* RpcError -> ErrorResponse

* Merge types::client into core::client

* Continued move types -> core

* Removing features to make checks pass

* Move rpc_module tests to tests crate

* Fixed doc comment links

* Add futures-util dependency for client

* Remove dead code

Co-authored-by: Niklas Adolfsson <[email protected]>

* fmt

* Feature gate the Client trait

* Move `Client` traits to `client` module

Co-authored-by: Niklas Adolfsson <[email protected]>

* rpc module: refactor calls/subs without a server

* Update utils/src/server/rpc_module.rs

* Update utils/src/server/rpc_module.rs

* Update utils/src/server/rpc_module.rs

* rpc module: add raw_json_requests + rename test_subscribe

* Update utils/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* add rustdoc examples

* fix tabs

* no more tabs in docs

* address grumbles

* Update utils/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* Squashed MethodSink

* Middleware WIP

* Passing all the information through

* Unnecessary `false`

* Apply suggestions from code review

Co-authored-by: David <[email protected]>

* Add a setter for middleware (#577)

* Fix try-build tests

* Add a middleware setter and an example

* Actually add the example

* Grumbles

* Use an atomic

* Set middleware with a constructor instead

* Resolve a todo

* Update ws-server/src/server.rs

Co-authored-by: Maciej Hirsz <[email protected]>

* Update ws-server/src/server.rs

Co-authored-by: Maciej Hirsz <[email protected]>

* Update ws-server/src/server.rs

Co-authored-by: Maciej Hirsz <[email protected]>

Co-authored-by: Maciej Hirsz <[email protected]>

* Middleware::on_response for batches

* Middleware in HTTP

* fmt

* Server builder for HTTP

* Use actual time in the example

* HTTP example

* Middleware to capture method not found calls

* An example of adding multiple middlewares. (#581)

* Add an example of adding multiple middlewares.

* Update examples/multi-middleware.rs

Co-authored-by: Maciej Hirsz <[email protected]>

* Update examples/Cargo.toml

Co-authored-by: Maciej Hirsz <[email protected]>

Co-authored-by: Maciej Hirsz <[email protected]>

* Move `Middleware` to jsonrpsee-types (#582)

* Move `Middleware` to jsonrpsee-types

* Move Middleware trait to jsonrpsee-types

* Add some docs.

* Link middleware to `with_middleware` methods in docs

* Doctests

* Doc comment fixed

* Clean up a TODO

* Switch back to `set_middleware`

* fmt

* Tests

* Add `on_connect` and `on_disconnect`

* Add note to future selves

Co-authored-by: David <[email protected]>

* feat: override `method` subscription notif

* Arrow syntax for overwrites (#569)

* check that unique notifs are used

* check that custom sub name is unique

* cargo fmt

* address grumbles

* Update proc-macros/src/rpc_macro.rs

* commit added tests

* Update proc-macros/src/render_server.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/render_server.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/rpc_macro.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/rpc_macro.rs

Co-authored-by: David <[email protected]>

* Update utils/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* grumbles

* fix long lines

* Update utils/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* Update utils/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/rpc_macro.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/render_server.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/render_server.rs

Co-authored-by: David <[email protected]>

* more grumbles

Co-authored-by: Maciej Hirsz <[email protected]>
Co-authored-by: David <[email protected]>

* rpc module: report error on invalid subscription

* fix tests

* remove some boiler plate

* remove unused code

* fix: enable several sockaddress when starting servers

* nits

* more verbose asserts in rustdoc tests

* fix tests

* fix tests again

* Fix some clippy issues

* Add an interval to periodically wake the SelectDriver Waker

* Apply suggestions from code review

Co-authored-by: David <[email protected]>

* Naming grumbles

Co-authored-by: David <[email protected]>

* ws server: don't kill connection max limit exceeds

* Update ws-server/src/server.rs

* actually use max size in soketto

* rewrite me

* improve logs

* use soketto fix

* rewrite me

* fix nit

* revert unintentional change

* use soketto 0.7.1

* fix logger

* Update ws-server/src/server.rs

Co-authored-by: David <[email protected]>

* Update ws-server/src/server.rs

Co-authored-by: David <[email protected]>

* Update ws-server/src/server.rs

Co-authored-by: David <[email protected]>

* Update ws-server/src/server.rs

Co-authored-by: David <[email protected]>

* Update ws-server/src/server.rs

Co-authored-by: David <[email protected]>

* fix build

* reject too large response

* fix some DRY code

* feat: bounded serializer for RpcModule

* Update utils/src/server/helpers.rs

Co-authored-by: Maciej Hirsz <[email protected]>

* Update utils/src/server/helpers.rs

Co-authored-by: Maciej Hirsz <[email protected]>

* address grumbles: safety and other nits

* address grumbles: MaxCallSize on closures instead

* Update utils/src/server/helpers.rs

Co-authored-by: David <[email protected]>

* use max response size on errors too

* Revert "use max response size on errors too"

This reverts commit 3b07e42d257b2eebae311b92b7f72594d94d5f87.

* include max limit in error response

Co-authored-by: David <[email protected]>
Co-authored-by: Maciej Hirsz <[email protected]>

* ws server: don't kill connection max limit exceeds

* Update ws-server/src/server.rs

* actually use max size in soketto

* rewrite me

* improve logs

* use soketto fix

* rewrite me

* fix nit

* revert unintentional change

* use soketto 0.7.1

* fix logger

* Update ws-server/src/server.rs

Co-authored-by: David <[email protected]>

* Update ws-server/src/server.rs

Co-authored-by: David <[email protected]>

* Update ws-server/src/server.rs

Co-authored-by: David <[email protected]>

* Update ws-server/src/server.rs

Co-authored-by: David <[email protected]>

* Update ws-server/src/server.rs

Co-authored-by: David <[email protected]>

* fix build

Co-authored-by: David <[email protected]>

* Implement Future for server handles

* Explicitly assert timeout errors in tests

* reproduce Kian's issue

* fix ws server wait until batches has completed

* fix nit

* clippify

* enable benches for ws batch requests

* use stream instead of futures::join_all

* clippify

* address grumbles: better assert

* Document resource limiting

* Formatting

* More formatting

* fmt

* Doc comment wording

Co-authored-by: David <[email protected]>

* Cross-reference docs between servers and utils

* Fix grumbles

* More elaborate description

* Update http-server/Cargo.toml

Co-authored-by: Niklas Adolfsson <[email protected]>

* Apply suggestions from code review

Co-authored-by: David <[email protected]>

* Allow for unlimited resources if cap is 0

* fmt

* Update ws-server/Cargo.toml

Co-authored-by: Niklas Adolfsson <[email protected]>

* Clarify 8 resources, 0s in costs/limits, and runtime errors

Co-authored-by: David <[email protected]>
Co-authored-by: Niklas Adolfsson <[email protected]>

* [ws server]: refactor start()

* remove needless clone

* fix http server too

* fmt

* unify stop APIs

* fix bad merge

* add API to use custom tokio runtime

* use futures oneshot

* fix unintentional change

* Revert "use futures oneshot"

This reverts commit 0432f6ba41d49a3494943458657a31f8afd79160.

* Update http-server/src/server.rs

Co-authored-by: Maciej Hirsz <[email protected]>

Co-authored-by: Maciej Hirsz <[email protected]>

* Squash post merge

* RAII claims and some comments

* Adding resources to Methods

* Use arrays for claiming all resources at once

* Register resources on ws-server

* Return MethodResourcesBuilder from register_method

* Return MethodResourcesBuilder from register_async_method

* fmt and comments

* Add `verify_and_insert` to get rid of `or_insert` in order to get mut ref to value

* Resources initialization

* No need for ResourceBuilder

* Comment all the things

* Handle errors on ws-server `start` method

* Passing ResourceGuard to method execution

* Start ws-server in background, simplify all the tests!

* Claiming resources on runtime

* Do checked math on resource claiming

* Clearer error path for resources at limit

* Resource limiting tests WIP

* Unbox async futures

* fmt

* Cleaner proc macro

* Fix lifetime issues with ResourceGuard with internal Arc

* Remove unnecessary lifetime generic

* Working tests

* Use turbofish in tests

* fmt

* Fixed (?) benches

* Apply suggestions from code review

Co-authored-by: David <[email protected]>

* Tweak more grumbles

* Adding a test for resource limiting with proc macros (WIP)

* Reworking attribute macros for resource limiting (#507)

* Squashed macros WIP

* bae-less macros!

* Make things simpler and more readable

* Some comments and DRY aliases parsing

* Naming things is hard

* Respan is no longer needed

* Simpler Arguments

* Remove stale code

* Apply suggestions from code review

Co-authored-by: Niklas Adolfsson <[email protected]>

* syn-up all the things, handle resources on methods

Co-authored-by: Niklas Adolfsson <[email protected]>

* Resource limiting on the http server

* Test two resources at once

* Test the HTTP server for resource limiting

* Apply suggestions from code review

Co-authored-by: David <[email protected]>
Co-authored-by: Niklas Adolfsson <[email protected]>

* Comment tweaks

* More explicit error type for `AttributeMeta::retain`

* Remove generic from ResourceTable; derive Default on ws Builder

* Make the macros shorter

Co-authored-by: David <[email protected]>
Co-authored-by: Niklas Adolfsson <[email protected]>

* feat(ws client): support redirections

* reuse socket

* reuse socket

* add hacks

* fix build

* remove hacks

* fix bad merge

* address grumbles

* fix grumbles

* fix grumbles

* fix nit

* add redirection test

* Update test-utils/src/types.rs

* Resolved todo

* Check that redirected client actually works

* Rename test-utils "types" to "mocks"

* Fix windows test (?)

* fmt

* What is wrong with you windows?

* Ignore redirect test on windows

* fix bad transport errors

* debug windows tests

* update soketto

* maybe fix windows test

* add config flag for max redirections

* revert faulty change.

Relative reference must start with either `/` or `//`

* revert windows path

* use manual join paths

* remove url dep

* Update ws-client/src/tests.rs

* default max redirects 5

* remove needless clone vec

* fix bad merge

* cmon CI run

Co-authored-by: David Palm <[email protected]>

* Unbox async futures

* fmt

* Cleaner proc macro

* Remove unnecessary parens around async blocks

* fix: server should not send to closed subscription

Closing #481

* add a test

* rewrite me

* remove faulty server close

* fix tests

* fix tests

* remove needless code

* Update ws-client/src/client.rs

* fix: remove atomic flag; use `close_sender`

* remove terminate message; use close channel

* remove faulty subscription closed error msg

* fix tests

* fix grumbles

* fix build

* Re-export v2::params, error, request and response types

* Use shallow type paths

* Renames

* More renames

* fmt

* s/RpcParams/Params/

* Add a type alias `SubscriptionResponse` to `Notification` which is parametrized to `SubscriptionPayload`
Move `SubscriptionResponse` to the `response` module

* Fix doc links

* Cleanup and some docs

* Review feedback

* cleanup

* Silence warning on cors_max_age

* Revert change to `AccessControl`

* Add a test illustrating how to use the `call` convenience method

* Extend test + review feedback

* log

* log

* log

* log

* log

* log

* log

* log

* log

* log

* log

* log

* log

* log

* log

* Add Methods::test_subscription
Log more when params parsing fails

* Add call_with test helper (ty @niklas!) + cleanup

* Remove todo (part of https://github.com/paritytech/jsonrpsee/issues/457

)

* fmt

* Let `test_subscription` be called from other crates

* Manually fix indentation

* fmt

* SSself-review grumbles

* CallError::InvalidParams carries an anyhow::Error

* fmt

* Tweak docs

* Update utils/src/server/rpc_module.rs

Co-authored-by: Niklas Adolfsson <[email protected]>

* review grumble

* Fix todos

* fmt

* Fixup error messages

* Include source in the error message for `CallError`

* fmt

* Update proc-macros/src/render_server.rs

Co-authored-by: Maciej Hirsz <[email protected]>

* Mention needing jsonrpsee crate in scope

* Resolve todo

* Impl ToRpcParams for 0-sized array

* optimized logging

Co-authored-by: Niklas Adolfsson <[email protected]>
Co-authored-by: Maciej Hirsz <[email protected]>

* Reject overflowing connection with status code 429

* fmt

* rename Handshake -> HandshakeMode for clarity; verbose test

* Gracefully shutdown after rejecting to hopefully fix the errors on windows

* HandshakeMode -> HandshakeResponse; tweak pending subscriptions on shutdown test