Update release documentation + checklist (#2079)

* add instructions for extrinsic verification

* update release documentation

polkadot/.github/ISSUE_TEMPLATE/release.md

@@ -8,23 +8,31 @@ title: Polkadot {{ env.VERSION }} Release checklist
 This is the release checklist for Polkadot {{ env.VERSION }}. **All** following
 checks should be completed before publishing a new release of the
 Polkadot/Kusama/Westend runtime or client. The current release candidate can be
-checked out with `git checkout {{ env.VERSION }}`
+checked out with `git checkout release-{{ env.VERSION }}`
 
 ### Runtime Releases
 
+These checks should be performed on the codebase prior to forking to a release-
+candidate branch.
+
 - [ ] Verify [`spec_version`](#spec-version) has been incremented since the
     last release for any native runtimes from any existing use on public
     (non-private/test) networks.
-- [ ] Verify [new migrations](#new-migrations) complete successfully, and the
-    runtime state is correctly updated.
 - [ ] Verify previously [completed migrations](#old-migrations-removed) are
-    removed.
+    removed for any public (non-private/test) networks.
 - [ ] Verify pallet and [extrinsic ordering](#extrinsic-ordering) has stayed
     the same. Bump `transaction_version` if not.
 - [ ] Verify new extrinsics have been correctly whitelisted/blacklisted for
     [proxy filters](#proxy-filtering).
 - [ ] Verify [benchmarks](#benchmarks) have been updated for any modified
     runtime logic.
+
+The following checks can be performed after we have forked off to the release-
+candidate branch.
+
+- [ ] Verify [new migrations](#new-migrations) complete successfully, and the
+    runtime state is correctly updated for any public (non-private/test)
+    networks.
 - [ ] Verify [Polkadot JS API](#polkadot-js) are up to date with the latest
     runtime changes.
 
@@ -59,7 +67,8 @@ Add any necessary assets to the release. They should include:
 
 The release notes should list:
 
-- The priority of the release (i.e., how quickly users should upgrade)
+- The priority of the release (i.e., how quickly users should upgrade) - this is
+    based on the max priority of any *client* changes.
 - Which native runtimes and their versions are included
 - The proposal hashes of the runtimes as built with
     [srtool](https://gitlab.com/chevdor/srtool)
@@ -77,16 +86,17 @@ A runtime upgrade must bump the spec number. This may follow a pattern with the
 client release (e.g. runtime v12 corresponds to v0.8.12, even if the current
 runtime is not v11).
 
+### Old Migrations Removed
+
+Any previous `on_runtime_upgrade` functions from old upgrades must be removed
+to prevent them from executing a second time. The `on_runtime_upgrade` function
+can be found in `runtime/<runtime>/src/lib.rs`.
+
 ### New Migrations
 
 Ensure that any migrations that are required due to storage or logic changes
 are included in the `on_runtime_upgrade` function of the appropriate pallets.
 
-### Old Migrations Removed
-
-Any previous `on_runtime_upgrade` functions from old upgrades must be removed
-to prevent them from executing a second time.
-
 ### Extrinsic Ordering
 
 Offline signing libraries depend on a consistent ordering of call indices and
@@ -94,6 +104,23 @@ functions. Compare the metadata of the current and new runtimes and ensure that
 the `module index, call index` tuples map to the same set of functions. In case
 of a breaking change, increase `transaction_version`.
 
+To verify the order has not changed:
+
+1. Download the latest release-candidate binary either from the draft-release
+on Github, or
+[AWS](https://releases.parity.io/polkadot/x86_64-debian:stretch/{{ env.VERSION }}-rc1/polkadot)
+(adjust the rc in this URL as necessary).
+2. Run the release-candidate binary using a local chain:
+`./polkadot --chain=polkadot-local` or `./polkadot --chain=kusama.local`
+3. Use [`polkadot-js-tools`](https://github.com/polkadot-js/tools) to compare
+the metadata:
+  - For Polkadot: `docker run --network host jacogr/polkadot-js-tools metadata wss://rpc.polkadot.io ws://localhost:9944`
+  - For Kusama: `docker run --network host jacogr/polkadot-js-tools metadata wss://kusama-rpc.polkadot.io ws://localhost:9944`
+4. Things to look for in the output are lines like:
+  - `[Identity] idx 28 -> 25 (calls 15)` - indicates the index for `Identity` has changed
+  - `[+] Society, Recovery` - indicates the new version includes 2 additional modules/pallets.
+  - If no indices have changed, every modules line should look something like `[Identity] idx 25 (calls 15)`
+
 Note: Adding new functions to the runtime does not constitute a breaking change
 as long as they are added to the end of a pallet (i.e., does not break any
 other call index).

polkadot/.github/workflows/publish-draft-release.yml

@@ -139,5 +139,5 @@ jobs:
         with:
           room_id: ${{ secrets.INTERNAL_POLKADOT_MATRIX_ROOM_ID }}
           access_token: ${{ secrets.MATRIX_ACCESS_TOKEN }}
-          message: "**New version of polkadot tagged**: ${{ github.ref }}<br/>Gav: Draft release created: ${{ needs.publish-draft-release.outputs.release_url }}"
+          message: "**New version of polkadot tagged**: ${{ github.ref }}<br/>Draft release created: ${{ needs.publish-draft-release.outputs.release_url }}"
           server: "matrix.parity.io"

polkadot/.github/workflows/release-candidate.yml

@@ -45,7 +45,7 @@ jobs:
         if: steps.compute_tag.outputs.first_rc == 'true'
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          BRANCH: ${{ steps.compute_tag.outputs.version }}
+          VERSION: ${{ steps.compute_tag.outputs.version }}
         with:
           filename: .github/ISSUE_TEMPLATE/release.md
       - uses: s3krit/matrix-message-action@v0.0.2

polkadot/RELEASE.md

@@ -3,14 +3,14 @@ Polkadot Release Process
 
 ### Branches
 * release-candidate branch: The branch used for staging of the next release.
-  Named like `release-v0.8.26` 
+  Named like `release-v0.8.26`
 * release branch: The branch to which successful release-candidates are merged
   and tagged with the new version. Named literally `release`.
 
 ### Notes
 * The release-candidate branch *must* be made in the paritytech/polkadot repo in
 order for release automation to work correctly
-* Any new pushes/merges to the release-candidate branch (for example, 
+* Any new pushes/merges to the release-candidate branch (for example,
 refs/heads/release-v0.8.26) will result in the rc index being bumped (e.g., v0.8.26-rc1
 to v0.8.26-rc2) and new wasms built.
 
@@ -32,14 +32,25 @@ automated and require no human action.
   completed
 6. (optional) If a fix is required to the release-candidate:
   1. Merge the fix with `master` first
-  2. Checkout the release-candidate branch and merge `master`
-  3. Revert all changes since the creation of the release-candidate that are
-  **not** required for the fix.
-  4. Push the release-candidate branch to Github - this is now the new release-
+  2. Cherry-pick the commit from `master` to `release-v0.8.26`, fixing any
+  merge conflicts. Try to avoid unnecessarily bumping crates.
+  3. Push the release-candidate branch to Github - this is now the new release-
   candidate
+  4. Depending on the cherry-picked changes, it may be necessary to perform some
+  or all of the manual tests again.
 7. Once happy with the release-candidate, perform the release using the release
   script located at `scripts/release.sh` (or perform the steps in that script
   manually):
   - `./scripts/release.sh v0.8.26`
 8. NOACTION: The HEAD of the `release` branch will be tagged with `v0.8.26`,
-  and a final release will be created on Github.
\ No newline at end of file
+  and a final draft release will be created on Github.
+
+### Security releases
+
+Occasionally there may be changes that need to be made to the most recently
+released version of Polkadot, without taking *every* change to `master` since
+the last release. For example, in the event of a security vulnerability being
+found, where releasing a fixed version is a matter of some expediency. In cases
+like this, the fix should first be merged with master, cherry-picked to a branch
+forked from `release`, tested, and then finally merged with `release`. A
+sensible versioning scheme for changes like this is `vX.Y.Z-1`.