* feat: limit the number of subscriptions

Closing #729

* fix nit

* Update core/src/server/helpers.rs

* add integration tests + some fixes so it works

* cargo fmt

* fix doc links

* Unsubscribe calls should avoid subscription limits

Point to Tokio 1.16 (we use a method from it), and a little special treatment for unsubscribe methods

* No resource limiting for Unsubscribe calls

* Test that we can still unsubscribe after hitting a limit

* Fix a comment typo

Co-authored-by: Alexandru Vasile <[email protected]>

* Update core/src/server/rpc_module.rs

* Update core/src/server/rpc_module.rs

Co-authored-by: James Wilson <[email protected]>
Co-authored-by: Alexandru Vasile <[email protected]>

* fix: use https conn for https

* enable integration tests for https and wss

* remove socket options of http client

fix(rpc module): close subscription task when a subscription is `unsubscribed` via the `unsubscribe call` (#743)

* refactor: remove SubscriptionSink::inner_send

* fix: close running task if unsubscribed

* Update core/src/server/rpc_module.rs

* Update core/src/server/rpc_module.rs

* fix nits

* Update core/src/server/rpc_module.rs

* add test for canceling subscriptions

* print subscription info; once per minute

* revert closure stuff

* Revert "print subscription info; once per minute"

This reverts commit 366176a8

.

* use tokio::sync::watch instead of oneshot

The receiver is clonable and it's possible to check whether the sender is still alive

* Update tests/tests/helpers.rs

Co-authored-by: David <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* grumbles: use unwrap in tests

* add test for reuse pipe_from_stream

Co-authored-by: David <[email protected]>

* fix(rpc module): fail subscription with bad params

* draft; show my point

* fix tests

* fix build

* add tests for proc macros too

* add tests for bad params in proc macros

* fix nits

* commit all files

* add ugly fix for proc macro code

* add more user friendly API

* make SubscriptionSink::close take mut self

* fix grumbles

* show james some code

* Update core/src/server/rpc_module.rs

Co-authored-by: James Wilson <[email protected]>

* remove needless clone

* fix build

* client fix docs + error type

* simplify code: merge connect reset and unsubscribe close reason

* remove unknown close reason

* refactor: remove Error::SubscriptionClosed

* add some nice APIs to ErrorObjectOwned

* unify api

* address grumbles

* remove redundant methods for close and reject

* proc macro: compile err when subscription -> Result

* rpc module: fix test subscription test

* Update core/src/server/rpc_module.rs

Co-authored-by: James Wilson <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: James Wilson <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: James Wilson <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: James Wilson <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: James Wilson <[email protected]>

* Update proc-macros/src/lib.rs

Co-authored-by: James Wilson <[email protected]>

* address grumbles

* remove faulty comment

* Update core/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* Update core/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* fix: don't send `RPC Call failed: error`.

* remove debug assert

Co-authored-by: James Wilson <[email protected]>
Co-authored-by: David <[email protected]>

* fix(ws server): close all subscriptions

* fix test

* better comment and longer period

* revert unintentional change

* unify servers; use tokio::TcpListener

* add API to build server from tcp socket

* build_from_tcp nit: not async

* fix nits

* change `build_from_tcp` force users configure hyper

The hyper settings might contradict to settings on the provided
socket, force users of this API to configure that avoid confusion and
unexpected settings.

* Update http-server/src/server.rs

* fix doc links

* add separate APIs `build_from_tcp` and `build_from_hyper`

* fix clippy

* remove dead code

* fix build

* fix nit in example code

* fix(client): close subscription when server sent `SubscriptionClosed` notification

* Update core/src/client/async_client/helpers.rs

* refactor: make `pipe_from_stream` take stream of result

The rationale for this is that it is more flexible for use cases when `Stream<Item = Result<T, Error>>`.

Take for example `tokio_stream::Broadcast` then one would have to something like:

```rust
   let stream = BroadcastStream::new(rx).take_while(|r| future::ready(r.is_ok())).filter_map(|r| future::ready(r.ok()));
```

Of course it's a bit awkward to return `Result` when the underlying stream can't fail but I think that's fair trade-off
here.

* Update core/src/server/rpc_module.rs

Co-authored-by: Tarik Gul <[email protected]>

* pipe_from_stream: make E: Display instead of StdError

* add a test

* add `pipe_from_try_stream` API to support `TryStream`

* Update tests/tests/integration_tests.rs

* Update proc-macros/src/lib.rs

Co-authored-by: Tarik Gul <[email protected]>

* refactor: get rid off `Error::Request` variant

* fix nit

* to_owned -> to_call_error

* wip wip wip

Use tokio::sync::Notify to signal to the server when a subscriber has gone away without calling unsubscribe

* Cleanup

* Fmt

* More cleanup more TODOs

* fmt

* Address a few todos

* Update core/src/server/rpc_module.rs

Co-authored-by: Niklas Adolfsson <[email protected]>

* Update ws-server/src/server.rs

Co-authored-by: Niklas Adolfsson <[email protected]>

* Fix bad merge

* Test

* fmt

* fix test

* Finish test

* Cleanup
Add a second subscription to serverless test

* Update tests/tests/integration_tests.rs

Co-authored-by: Niklas Adolfsson <[email protected]>

* simplify test

* Review feedback: avoid allocations

* cleanup

* Remove async-channel

* remove async-channel deps

Co-authored-by: Niklas Adolfsson <[email protected]>

* [proc macros]: only generate unsub if not provided

* Update proc-macros/src/rpc_macro.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/lib.rs

Co-authored-by: David <[email protected]>

* fix grumbles

* Update proc-macros/src/lib.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/lib.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/lib.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/tests/ui/correct/alias_doesnt_use_namespace.rs

Co-authored-by: David <[email protected]>

Co-authored-by: David <[email protected]>

* feat(client): support request id as Strings.

* add tests for Id::String

* address grumbles: move id_kind to RequestManager

* Update client/http-client/src/client.rs

* types: take ref to `ID` get rid of some `Clone`

* remove more clone

* grumbles: rename tests

Add a method to RpcModule that transforms the module into a `RpcModule<()>`, i.e. removes the context. (#660)

* Add a method to RpcModule, `decontextualize`, that transforms the module into a `RpcModule<()>`, i.e. removes the context.

* Merging a module with an empty module cannot fail

* fmt

* Address grumbles

* Cleanup macro

* Update core/src/server/rpc_module.rs

Co-authored-by: Maciej Hirsz <[email protected]>

Co-authored-by: Maciej Hirsz <[email protected]>

* feat(rpc module): add_stream to subscription sink

* fix some nits

* unify parameters to rpc methods

* Update core/src/server/rpc_module.rs

* Update tests/tests/integration_tests.rs

Co-authored-by: David <[email protected]>

* address grumbles

* fix subscription tests

* new type for `SubscriptionCallback` and glue code

* remove unsed code

* remove todo

* add missing feature tokio/macros

* make `add_stream` cancel-safe

* rename add_stream and return status

* fix nits

* rename stream API -> streamify

* Update core/src/server/rpc_module.rs

* provide proper close reason

* spelling

* consume_and_streamify + docs

* fmt

* rename API pipe_from_stream

* improve logging; indicate which subscription method that failed

Co-authored-by: David <[email protected]>

* fix(http server): handle post and option requests.

* Update http-server/src/response.rs

* fmt

* grumbles: validate CORS before checking method kind

* revert unintentional change

* Add A few tests around CORS and http method/header acceptance

* Tweak the CORS test

* return appropriate headers for CORS preflight response

* cargo fmt

* jsonrpc, not jsonrpsee in test requests

* don't test OPTIONS method in unsupported_methods test as it has different expectations

* Update tests/tests/integration_tests.rs

* Update tests/tests/integration_tests.rs

* Update tests/tests/integration_tests.rs

* Update tests/tests/integration_tests.rs

* Add an example to test CORS in browser, and a couple of fixes to CORS handling

* cargo fmt

* Update examples/cors_server.rs

Co-authored-by: James Wilson <[email protected]>

* PoC

* introduce IdProvider trait

* revert Cow stuff

* Update ws-server/src/server.rs

* fix tests

* cargo fmt

* use 'static lifetime in SubscriptionId

* fix tests

* fmt

* make tests compile again

* fix tests

* Fix tests

* Move IdProvider impls to own module

* move `sub-id gen` types to `core`

* fix doc links

* make rand non-optional dep

* feature gate: id provider

Co-authored-by: David Palm <[email protected]>

* extract ws client to async client

* bring back jsonrpsee-ws-client crate

* new crate core client

* add missing file

* jsonrpsee crate: add core client

* rexport core client

* more re-exports

* downgrade trait bounds

* update version

* fix nits

* send close reason

* Update types/src/traits.rs

* move 'TEN_MB_CONST' to core

* Update client/core-client/Cargo.toml

Co-authored-by: David <[email protected]>

* Update client/core-client/src/lib.rs

Co-authored-by: David <[email protected]>

* cargo fmt

* Update client/ws-client/src/lib.rs

Co-authored-by: David <[email protected]>

* Update client/ws-client/src/lib.rs

Co-authored-by: David <[email protected]>

* move `async-client` core

The `async-client` is hidden behind a new feature flag `async-client`
because it brings in additional dependecies such as tokio rt.

* fix docs

* add example how to use "core client"

* fix build

* Update http-server/Cargo.toml

* Update client/transport/Cargo.toml

Co-authored-by: David <[email protected]>

* Nuke V2

* fmt

* Formatting and imports

* Updated benches

* Fix doc comment link

* Brace imports in ws-server

* Reworking imports

* std first

* fmt

* std on top

* Update to match changed line numbers

* Rename jsonrpsee_utils -> jsonrpsee_core

* Migrating things types -> core

* RpcError -> ErrorResponse

* Merge types::client into core::client

* Continued move types -> core

* Removing features to make checks pass

* Move rpc_module tests to tests crate

* Fixed doc comment links

* Add futures-util dependency for client

* Remove dead code

Co-authored-by: Niklas Adolfsson <[email protected]>

* fmt

* Feature gate the Client trait

* Move `Client` traits to `client` module

Co-authored-by: Niklas Adolfsson <[email protected]>

* refactor `SubscriptionClosed`

I noticed that we "trying" to sending "close messages" on subscriptions
that already has been terminated by the client which is unnecessary.

Moreover, this introduces `fn is_closed` to check whether a given
channel is closed without having to send an actual message on the channel.

* cargo fmt

* Update types/src/error.rs

Co-authored-by: David <[email protected]>

* fix nits

* more grumbles

* make rust stable happy

* SubscriptionClosed: add deny unknown fields

Co-authored-by: David <[email protected]>

* rpc module: refactor calls/subs without a server

* Update utils/src/server/rpc_module.rs

* Update utils/src/server/rpc_module.rs

* Update utils/src/server/rpc_module.rs

* rpc module: add raw_json_requests + rename test_subscribe

* Update utils/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* add rustdoc examples

* fix tabs

* no more tabs in docs

* address grumbles

* Update utils/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* get the tests passing

* cargo fmt

* tweak comment wording

* point to StreamExt in the next() method impl just so that people know to import it if desired

* ignore clippy lint on next() Iterator-like method

* Fix an example

* actually, unwrap instead of transpose

* clients: introduce tls feature flag

* Update tests/tests/integration_tests.rs

* fix: don't rebuild tls connector of every connect

* fix tests + remove url dep

* fix tests again

* Squashed MethodSink

* Middleware WIP

* Passing all the information through

* Unnecessary `false`

* Apply suggestions from code review

Co-authored-by: David <[email protected]>

* Add a setter for middleware (#577)

* Fix try-build tests

* Add a middleware setter and an example

* Actually add the example

* Grumbles

* Use an atomic

* Set middleware with a constructor instead

* Resolve a todo

* Update ws-server/src/server.rs

Co-authored-by: Maciej Hirsz <[email protected]>

* Update ws-server/src/server.rs

Co-authored-by: Maciej Hirsz <[email protected]>

* Update ws-server/src/server.rs

Co-authored-by: Maciej Hirsz <[email protected]>

Co-authored-by: Maciej Hirsz <[email protected]>

* Middleware::on_response for batches

* Middleware in HTTP

* fmt

* Server builder for HTTP

* Use actual time in the example

* HTTP example

* Middleware to capture method not found calls

* An example of adding multiple middlewares. (#581)

* Add an example of adding multiple middlewares.

* Update examples/multi-middleware.rs

Co-authored-by: Maciej Hirsz <[email protected]>

* Update examples/Cargo.toml

Co-authored-by: Maciej Hirsz <[email protected]>

Co-authored-by: Maciej Hirsz <[email protected]>

* Move `Middleware` to jsonrpsee-types (#582)

* Move `Middleware` to jsonrpsee-types

* Move Middleware trait to jsonrpsee-types

* Add some docs.

* Link middleware to `with_middleware` methods in docs

* Doctests

* Doc comment fixed

* Clean up a TODO

* Switch back to `set_middleware`

* fmt

* Tests

* Add `on_connect` and `on_disconnect`

* Add note to future selves

Co-authored-by: David <[email protected]>

* feat: override `method` subscription notif

* Arrow syntax for overwrites (#569)

* check that unique notifs are used

* check that custom sub name is unique

* cargo fmt

* address grumbles

* Update proc-macros/src/rpc_macro.rs

* commit added tests

* Update proc-macros/src/render_server.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/render_server.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/rpc_macro.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/rpc_macro.rs

Co-authored-by: David <[email protected]>

* Update utils/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* grumbles

* fix long lines

* Update utils/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* Update utils/src/server/rpc_module.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/rpc_macro.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/render_server.rs

Co-authored-by: David <[email protected]>

* Update proc-macros/src/render_server.rs

Co-authored-by: David <[email protected]>

* more grumbles

Co-authored-by: Maciej Hirsz <[email protected]>
Co-authored-by: David <[email protected]>

Closes https://github.com/paritytech/jsonrpsee/issues/448

This PR adds an implementation for `SubscriptionClient` to the `HttpClient` struct, which makes it possible for http clients to use macro-generated RPC servers. If an http client tries to set up a subscription it will fail with a `HttpNotImplemented` error.

* fix: enable several sockaddress when starting servers

* nits

* more verbose asserts in rustdoc tests

* fix tests

* fix tests again

* Fix some clippy issues

* Add an interval to periodically wake the SelectDriver Waker

* Apply suggestions from code review

Co-authored-by: David <[email protected]>

* Naming grumbles

Co-authored-by: David <[email protected]>

* Array syntax aliases

* fmt

* Implement Future for server handles

* Explicitly assert timeout errors in tests

* reproduce Kian's issue

* fix ws server wait until batches has completed

* fix nit

* clippify

* enable benches for ws batch requests

* use stream instead of futures::join_all

* clippify

* address grumbles: better assert

* fix http client bench with request limit

* benches for jsonrpc servers

* workaround; dont use max request limit

* add subscriptions

* revert unintentional change

* ignore batch request bench for ws

* fmt

* log -> tracing

* test bench CI

* test bench v0.3

* wtf; run CI

* work plz

* remove test CI bench

* fix compile warn on macos

* Re-introduce object param parsing

* fmt and unnecessary collect

* register_blocking_method

* Support `blocking` flag in `#[method]`

* Macro support and tests

* MacOS CI fails if there are too many blocking tasks

* conn_id is never used in async callbacks

* Dry-run blocking callbacks to maybe satisfy MacOS CI

* Make tokio optional behind server feature flag

* Document `blocking` flag use in `#[method]`

* Comment why the test is disabled on mac

* Fix warnings on macos tests

* [ws server]: refactor start()

* remove needless clone

* fix http server too

* fmt

* unify stop APIs

* fix bad merge

* add API to use custom tokio runtime

* use futures oneshot

* fix unintentional change

* Revert "use futures oneshot"

This reverts commit 0432f6ba41d49a3494943458657a31f8afd79160.

* Update http-server/src/server.rs

Co-authored-by: Maciej Hirsz <[email protected]>

Co-authored-by: Maciej Hirsz <[email protected]>

* Remove unused import

* Add a simple test for angle braces in arguments

* Argument parsing should permit commas inside angle brackets

* Add a comment, move test to trybuild

* fix(ParamsSer): serialize to empty array

Serialize `ParamsSer::NoParams` to an empty array inorder to comply with the jsonrpc v2 spec.

* fix tests

* remove ParamsSer::NoParams

* address grumbles: adjust rpc params macro

* Squash post merge

* RAII claims and some comments

* Adding resources to Methods

* Use arrays for claiming all resources at once

* Register resources on ws-server

* Return MethodResourcesBuilder from register_method

* Return MethodResourcesBuilder from register_async_method

* fmt and comments

* Add `verify_and_insert` to get rid of `or_insert` in order to get mut ref to value

* Resources initialization

* No need for ResourceBuilder

* Comment all the things

* Handle errors on ws-server `start` method

* Passing ResourceGuard to method execution

* Start ws-server in background, simplify all the tests!

* Claiming resources on runtime

* Do checked math on resource claiming

* Clearer error path for resources at limit

* Resource limiting tests WIP

* Unbox async futures

* fmt

* Cleaner proc macro

* Fix lifetime issues with ResourceGuard with internal Arc

* Remove unnecessary lifetime generic

* Working tests

* Use turbofish in tests

* fmt

* Fixed (?) benches

* Apply suggestions from code review

Co-authored-by: David <[email protected]>

* Tweak more grumbles

* Adding a test for resource limiting with proc macros (WIP)

* Reworking attribute macros for resource limiting (#507)

* Squashed macros WIP

* bae-less macros!

* Make things simpler and more readable

* Some comments and DRY aliases parsing

* Naming things is hard

* Respan is no longer needed

* Simpler Arguments

* Remove stale code

* Apply suggestions from code review

Co-authored-by: Niklas Adolfsson <[email protected]>

* syn-up all the things, handle resources on methods

Co-authored-by: Niklas Adolfsson <[email protected]>

* Resource limiting on the http server

* Test two resources at once

* Test the HTTP server for resource limiting

* Apply suggestions from code review

Co-authored-by: David <[email protected]>
Co-authored-by: Niklas Adolfsson <[email protected]>

* Comment tweaks

* More explicit error type for `AttributeMeta::retain`

* Remove generic from ResourceTable; derive Default on ws Builder

* Make the macros shorter

Co-authored-by: David <[email protected]>
Co-authored-by: Niklas Adolfsson <[email protected]>

* Move RequestIdGuard to types::client

* Use the RequestIdGuard type for the http client as well

* Fix batch request ID handling (one batch uses one slot)

* Add a few tests to check that max_concurrent_requests work for http clients

* Docs