Skip to content
  1. Oct 17, 2023
  3. Oct 16, 2023
  5. Oct 15, 2023
  7. Oct 14, 2023
    • Julian Eager's avatar
      Discard `Executor` (#1855) · 9f7656df
      Julian Eager authored 
      

closes #622 

Pros:
* simpler interface, just functions:
`create_runtime_from_artifact_bytes()` and `execute_artifact()`

Cons:
* extra overhead of constructing executor semantics each time

I could make it a combination of
* `create_runtime_config(params)` (such that we could clone the
constructed semantics)
* `create_runtime(blob, config)`
* `execute_artifact(blob, config, params)`

Not sure if it's worth it though.

---------

Co-authored-by: default avatarBastian Köcher <[email protected]>
      9f7656df
    • juangirini's avatar
      Macros to use path instead of ident (#1474) · 7c87d61f
      juangirini authored
      7c87d61f
  9. Oct 13, 2023
  11. Oct 12, 2023
  13. Oct 11, 2023
  15. Oct 10, 2023
    • Sam Johnson's avatar
      upgrade to macro_magic 0.4.3 (#1832) · 5adcb3e1
      Sam Johnson authored 
      # Description

Upgrades `macro_magic` to 0.4.3, which introduces the ability to have
`export_tokens` use the same name as the underlying item for its
auto-generated macro name. Ultimately this will allow for better dev ux
in our derive_impl feature.
      5adcb3e1
    • Keith Yeung's avatar
      Use safe math when pruning statuses (#1835) · 3f5edc52
      Keith Yeung authored 
      

Co-authored-by: default avatarFrancisco Aguirre <[email protected]>
      3f5edc52
    • Liam Aharon's avatar
      remote-ext: fix state download stall on slow connections and reduce memory usage (#1295) · 55f35442
      Liam Aharon authored 
      Original PR https://github.com/paritytech/substrate/pull/14746

---

## Fixing stall

### Introduction
I experienced an apparent stall downloading state from
`https://rococo-try-runtime-node.parity-chains.parity.io:443` which was
having networking difficulties only responding to my JSONRPC requests
with 50-200KB/s of bandwidth.

This PR fixes the issue causing the stall, and generally improves
performance remote-ext when it downloads state by greatly reducing the
chances of a timeout occuring.

### Description
Introduces a new `REQUEST_DURATION_TARGET` constant and modifies
`get_storage_data_dynamic_batch_size` to

- Increase or decrease the batch size of the next request depending on
whether the elapsed time of the last request was gt or lt the target
- Reset the batch size to 1 if the request times out

This fixes an issue on slow connections that can otherwise cause
multiple timeouts and a stalled download when:

1. The batch size increases rapidly as remote-ext downloads keys with
small associated storage values
2. remote-ext tries to process a large series of subsequent keys all
with extremely large associated storage values (Rococo has a series of
keys 1-5MB large)
3. The huge storage values download for 5 minutes until the request
times out
4. The partially downloaded keys are thrown out and remote-ext tries
again with a smaller batch size, but the batch size is still far too
large and takes 5 minutes to be reduced again
5. The download will be essentially stalled for many hours while the
above step cycles


After this PR, the request size will

- Not grow as large to begin with, as it is regulated downwards as the
request duration exceeds the target
- Drop immediately to 1 if the request times out. A timeout indicates
the keys next in line to download have extremely large storage values
compared to previously downloaded keys, and we need to reset the batch
size to figure out what our new ideal batch size is. By not resetting
down to 1, we risk the next request timing out again.

## Reducing memory

As suggested by @bkchr, I adjusted `get_storage_data_dynamic_batch_size`
from being recursive to a loop which allows removing a bunch of clones
that were chewing through a lot of memory. I noticed actually it was
using up to 50GB swap previously when downloading Polkadot keys on a
slow connection, because it needed to recurse and clone a lot.

After this change it uses only ~1.5GB memory.
      55f35442