When testing `zombienet-sdk` with cross-chain calls with Asset Hub, we
found that the hrmp channels passed to the chainspec caused the
following issue:

`Error: "Invalid JSON blob: invalid type: map, expected a tuple of size
4 at line 1 column 2794"`

This is because the values of `HrmpChannelConfig` serialise to a map
instead of a tuple, as expected by
https://github.com/paritytech/polkadot-sdk/blob/fc906d5d0fb7796ef54ba670101cf37b0aad6794/polkadot/runtime/parachains/src/hrmp.rs#L492.
Tests were updated accordingly to facilitate this fix.

Additional commits were also added to enable some of Asset Hub on
Polkadot functionality observed in the classic Zombienet repo, so that
we are able to launch a local Polkadot network with Asset Hub should
anyone require. Example configuration files can be found at the bottom
of https://github.com/r0gue-io/pop-cli/pull/278/files for anyone
interested, although that PR is still a work in progress and dependent
on this one being accepted.

PS - I added tests where I could, but additional tests might require
some minor refactoring to make it easier to test only the logic that is
changed by this PR, which I am not sure what the appetite is? An example
would be refactoring the file name generation in `keystore.rs` into its
own function to more easily test the outputs without the `scoped_fs`.

WIP

---------

Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>

Add: 
 - Bump subxt version
 - Change visibility for internal crates

Change exports.

@rzadp @Bullrich

This pr introduce the ability to set the `base_dir` used by the
`namespace`. This allow users to reuse the same node's directories
(config and database) and re-start the network from the latest state in
sequential runs.

Thx!

fix #218

cc @sandreim thanks for the suggestion :)

Firstly, thank you for an amazing library! The original zombienet is a
Polkadot staple and now being able to use it completely from Rust is a
game changer for us: as you may be aware, we have integrated
zombienet-sdk with
[pop-cli](https://github.com/paritytech/zombienet-sdk).

In doing so, we needed to add a few getters to the public api to be able
to present certain information to the user, as well we expose a little
more of the inner errors. This PR simply upstreams these changes.

It also includes two additional fixes, which may be better implemented
based on your guidance:
- collator args werent working for us when specified in a config file,
which seemed to be due to missing `--` in the args list provided by
calling functions.
- artifact path wasnt set when trying to build a chain spec. This may no
longer be required based on the changes in
https://github.com/paritytech/zombienet-sdk/commit/c7ce50c0

I am happy to add tests if you are open to these changes. 

Couple of fixes for docker provider and to use an external tool to build
the chain-spec.

docker support 

---------

Co-authored-by: Javier Viola <javier@parity.io>

Add:
 - logs methods in NetworkNode

- [k8s] fix `cmd` generator for collators (needs to use exposed ports).

based on #190 

cc: @s0me0ne-unkn0wn

Fix https://github.com/paritytech/zombienet-sdk/issues/169

- Add logic to retrieve available args from node before spawning the
full network using the output of the command with --help

Concatenating the command line option and its value doesn't really work.
The shell treats `--opt val` as a single argument `--opt val` instead of
`--opt` `val` and fails to execute. This PR fixes that by separating
options from their values.

Error example:
```
error: unexpected argument '--workers-path ~/.cargo/bin/workers-1.6' found

  tip: a similar argument exists: '--workers-path'
```

Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>

Add: 
  - async drop impl in k8s namespace / `noop` in native.
  - Modify example test.

Add:
 - Logic to use db_snapshot in both native/k8s providers
 - Small fix on chain_spec_path (for both native/k8s)
 - New example `db_snapshot`

Add small validation, mostly to ensure that the `image` is set in k8s.

fix #143

Fix #173 

We should also add a follow up issue to improve the docs.

cc: @s0me0ne-unkn0wn

- Add `custom_parchain_fs_prefix` parameter to `add_para` fn (on running
network), this allow to manage the fs layout and allow to add paras with
the same id.
- Minor clippy auto-fixes

cc: @metricaez

---------

Co-authored-by: Loris Moulin <45130584+l0r1s@users.noreply.github.com>

Add: Manual variant to `RegistrationStrategy`, allow to deploy the
collators _but_ not register the para.
Fix: `created_paths` in native provider
Fix: Use `rpc` port for ws connections (orchestrator)

This is a very simple gitlab CI manifest to just check the tag on this
pipeline schedule it on Zombienet k8s runner and is the SA inject to the
runners in privileged enough to run command within the cluster or not

---------

Co-authored-by: Javier Viola <javier@parity.io>

This PR introduces the Kubernetes Provider for the SDK, with the use of
kube.rs for common operation and a custom file server using axum to
transfer files back and forth the nodes.

https://github.com/paritytech/zombienet-sdk/issues/35

This also includes the refacto of the native provider to reflect changes
to the provider structure with weak references and simplifying the use
of abstractions

---------

Co-authored-by: Javier Viola <pepoviola@gmail.com>
Co-authored-by: Javier Viola <javier@parity.io>
Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>

This PR lists some issues and suggested fixes for them (feel free to
pick them up separately and close this PR):

In addition to issues outlined in
https://github.com/paritytech/zombienet-sdk/issues/117#issuecomment-1867941395,
here are some issues spotted when writing
https://github.com/paritytech/disabling-e2e-tests:
- runtime genesis patch is applied incorrectly (extra `/genesis` pointer
shouldn't be added)
- malus accepts
[subcommands](https://github.com/paritytech/polkadot-sdk/blob/4c0e0e071355c1048d75fba538c96c35ac743547/polkadot/zombienet_tests/functional/0008-dispute-old-finalized.toml#L25),
having a command with spaces not supported, so I added a subcommand
support
- some types need to be exported in order to be able to reuse
setup/helper functions across multiple tests
- test cleanup doesn't always work (zombie polkadot processes - not
fixed here)

---------

Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>

Testing with moonbeam and found that the resulting chainspec produced
using the `moonbase-local` chain develops the following issue after the
generated plain chainspec is updated with the relevant para id:

```
Error: Zombienet(SpawnerError(Provider error
Caused by:
    File generation failed: Error: Service(Other("Error parsing spec file: trailing characters at line 25 column 14"))
```

The initial chainspec built from the binary is fine, but once
https://github.com/paritytech/zombienet-sdk/blob/e60deb25/crates/orchestrator/src/generators/chain_spec.rs#L345-L349
runs the issue occurs, as seen in the the following diff:

```
--- moonbase-local-plain copy.json	2024-02-11 02:51:15
+++ moonbase-local-plain.json	2024-02-11 02:52:24
@@ -11,7 +11,7 @@
     "tokenSymbol": "UNIT"
   },
   "relayChain": "westend-local",
-  "paraId": 1000,
+  "paraId": 2000,
   "codeSubstitutes": {},
   "genesis": {
     "runtime": {
\ No newline at end of file
@@ -22,19 +22,19 @@
         "balances": [
           [
             "0xf24ff3a9cf04c71dbc94d0b566f7a27b94566cac",
-            1208925819614629174706176
+            1.2089258196146292e24
           ],
           [
             "0x3cd0a705a2dc65e5b1e1205896baa2be8a07c6e0",
-            1208925819614629174706176
+            1.2089258196146292e24
           ],
           [
             "0x798d4ba9baf0064ec19eb4f0a1a45785ae9d6dfc",
-            1208925819614629174706176
+            1.2089258196146292e24
           ],
           [
             "0x773539d4ac0e786233d90a233654ccee26a613d9",
-            1208925819614629174706176
+            1.2089258196146292e24
           ]
         ]
       },
\ No newline at end of file
@@ -45,7 +45,7 @@
         "multiplier": "8000000000000000000"
       },
       "parachainInfo": {
-        "parachainId": 1000
+        "parachainId": 2000
       },
       "ethereumChainId": {
         "chainId": 1280
\ No newline at end of file
@@ -503,19 +503,19 @@
         "candidates": [
           [
             "0xf24ff3a9cf04c71dbc94d0b566f7a27b94566cac",
-            1000000000000000000000
+            1e21
           ],
           [
             "0x3cd0a705a2dc65e5b1e1205896baa2be8a07c6e0",
-            1000000000000000000000
+            1e21
           ]
         ],
         "delegations": [],
         "inflationConfig": {
           "expect": {
-            "min": 100000000000000000000000,
-            "ideal": 200000000000000000000000,
-            "max": 500000000000000000000000
+            "min": 1e23,
+            "ideal": 2e23,
+            "max": 5e23
           },
           "annual": {
             "min": 40000000,
\ No newline at end of file
@@ -552,7 +552,7 @@
         "eligibleCount": 50
       },
       "crowdloanRewards": {
-        "fundedAmount": 3000000000000000000000000
+        "fundedAmount": 3e24
       },
       "authorMapping": {
         "mappings": [
\ No newline at end of file
@@ -611,6 +611,11 @@
           "0xf24ff3a9cf04c71dbc94d0b566f7a27b94566cac",
           "0x3cd0a705a2dc65e5b1e1205896baa2be8a07c6e0"
         ]
+      },
+      "aura": {
+        "authorities": [
+          "5HEL3iLyDyaqmfibHXAXVzyQq4fBqLCHGMEYxZXgRAuhEKXX"
+        ]
       }
     }
   }
```

The fix appears to be to enable the
[`arbitrary_precision`](https://github.com/serde-rs/json/blob/master/Cargo.toml#L75)
feature on `serde_json`, which can be seen at
https://github.com/paritytech/polkadot-sdk/pull/2987/files. Manually
tested and solves the issue for me.