Weekly Dev Update #90

2021-04-29 — 9 min read

THORChain Weekly Dev Update for Week 90, 20–29 Apr; Chaosnet Stability, Fixes, Ledger, Community Updates, XDEFI Integration

Summary

Low-Gas Gremlins

4 updates were rapidly pushed to MCCN in order to address issues that had cropped up. The most significant of which was low-gas errors, where Nodes would report low-gas on chains, then broadcast outbound transactions. On UTXO chains this generally isn’t a problem since the process of “ChildPaysForParent” allows once pending UTXO to be consumed again, bumping up the gas rate.

Ethereum does not have this process, leading to a situation where a vault can get “stuck”, the only way to bump up a transaction is to sign the exact transaction again, or cancel them all by doing a 0-eth tx back to self. This problem was exacerbated because Ethereum has regular empty blocks, which was causing GETH gas estimations to drop down to low or even 0, which would be reported to THORChain.

Around $100k in funds were lost due to this:
1) An outbound tx with a low gas-rate would be delegated to a node, which would sign and send out
2) Since it did not confirm within 300 blocks, (1) would be re-delegated back to Asgard, and the node slashed.
3) Asgard would then sign the transaction, but with the same gas rate as (1), causing it to get stuck.
4) Asgard would then re-schedule itself, causing multiple transactions to pile up unconfirmed.
5) At some point the gas rates would drop down again, causing (1), (3) and all the duplicates (4) to be processed.

The final solution which took several attempts to roll out:
1) Make reported Gas Rates more resilient to empty blocks, with a trailing MAX of 20 blocks
2) Add logic for a node to self-cancel pending transactions every 150 blocks
3) Add a safety where nodes do not sign an outbound tx if it finds pending transactions from itself
4) When re-delegating/rescheduling, use a refreshed gas rate

The team ultimately reimbursed slashed funds to nodes and some members of the community came forward and returned duplicated funds (and retained a 10% bounty). Since the mechanism of slashing causes 1.5x to be paid into the pools, all LPs in the system benefitted to the expense of the treasury.

Solvencies

While funds were lost to duplicate transactions and node slash events, no funds were lost from unknown errors or theft. The treasury ultimately suffered a loss of around $50k, taking into account returned funds, and this was to the benefit of ecosystem users. The project and community are comfortable with this, noting the “expect chaos” messaging and realisation that not everything can be found and tested on Testnets.

The project is grateful to node operators reporting errors, and community members reporting the duplicate transactions, noting that everyone, by nature of $RUNE, is aligned to see ultimate success. If you ever see something out of place, including duplicated transactions, reach out, you will retain a 10% bounty.

Solvency can be tracked in real-time:

https://thorchain.network/solvency-mainnet/

Updates

THORNODE UPDATE 0.45.1

This release contains the following fixes
1. [BUG] Asymmetric withdraw from RUNE side , when emit asset is less then NativeTransactionFee , it fail without revert pool balance. PR: https://gitlab.com/thorchain/thornode/-/merge_requests/1643
2. [BUG] ETH send unnecessary network fee tx to thornode. PR: https://gitlab.com/thorchain/thornode/-/merge_requests/1642
3. [BUG] Errata process , Need to revert the fund from asgard even the tx is not finalised. PR: https://gitlab.com/thorchain/thornode/-/merge_requests/1649
4. [BUG] Repeated Low-gas Funding of Yggdrasil. PR: https://gitlab.com/thorchain/thornode/-/merge_requests/1646 , https://gitlab.com/thorchain/thornode/-/merge_requests/1650 , https://gitlab.com/thorchain/thornode/-/merge_requests/1648
5. [BUG] Reduce TSS keysign timeout from 3 minutes to 35 seconds. PR: https://gitlab.com/thorchain/thornode/-/merge_requests/1652

UPDATE 0.46.0

This release contains the following fixes

1. [BUG] allow fuzzy asset matching in memos PR: https://gitlab.com/thorchain/thornode/-/merge_requests/1651
2. [BUG] normal node accumulate a lot slash point because a tx has been report as part of mempool scan , will report it again when block get commited , cause network to slash, PR : https://gitlab.com/thorchain/thornode/-/merge_requests/1653
3. [BUG] When prune bifrost block meta , make sure the self signed tx is not in mempool PR: https://gitlab.com/thorchain/thornode/-/merge_requests/1655
4. [BUG] on ETH chain , exclude yggdrasil outbound tx from confirmation counting. PR: https://gitlab.com/thorchain/thornode/-/merge_requests/1657

UPDATE 0.47.0

This release contains the following fixes
1. [bugfix] bond event can be off by 1 on first occasion PR: https://gitlab.com/thorchain/thornode/-/merge_requests/1666
2. [bugfix] update max gas when rescheduling a transactions PR: https://gitlab.com/thorchain/thornode/-/merge_requests/1665
3. [bugfix] Fix the issue that fee rate is not part of the consensus PR: https://gitlab.com/thorchain/thornode/-/merge_requests/1664
4. [bugfix] Make sure ETH return the highest gas price in the last 10 blocks PR: https://gitlab.com/thorchain/thornode/-/merge_requests/1662
5. [bugfix] swaps can get lost in certain scenarios PR: https://gitlab.com/thorchain/thornode/-/merge_requests/1661
6. [bugfix] IL on WithdrawBasisPoints , and No impermanent loss protection for staging pool PR: https://gitlab.com/thorchain/thornode/-/merge_requests/1660

UPDATE 0.48.0

This release contains the following fixes

1. [ADD] ETH chain client vault unstuck process , if a transaction signed and broadcast to ETH chain , didn’t commit after 150 THORChain blocks , it cancel it automatically. Also it extend the ETH client gas price cache to 20 blocks , it will choose the highest gas price in the last 20 blocks
PR: https://gitlab.com/thorchain/thornode/-/issues/901

Multichain Resources

Chaosnet

Midgard: https://midgard.thorchain.info/v2/doc

THORNode: https://thornode.thorchain.info/thorchain/doc/

RPC: https://rpc.thorchain.info(https://cosmos.network/rpc/v0.37.9)

Seed Service: https://seed.thorchain.info

Testnet

Midgard: https://testnet.midgard.thorchain.info/v2/doc

THORNode: https://testnet.thornode.thorchain.info/thorchain/doc/

RPC: https://testnet.rpc.thorchain.info(https://cosmos.network/rpc/v0.37.9)

Seed Service: https://testnet.seed.thorchain.info

THORNode

A lot of fixes to achieve stability, including some bugs around how pending liquidity is added and reported on.

https://gitlab.com/thorchain/thornode/-/merge_requests?scope=all&state=merged

Midgard

A lot of fixes to achieve stability. No major changes to report.

https://gitlab.com/thorchain/midgard/-/merge_requests?scope=all&state=merged

XChainJS

Stability and fixes for Asgardex requirements. If you are interested to help maintain the library, please reach out.

https://xchainjs.org

Bridges

Actively-developed bridges will be reported here going forward. Bridges are commissioned once a team who can implement is identified and work begins.

Complete

Bitcoin: Deployed to chaosnet
Ethereum: Deployed to chaosnet
BitcoinCash: Deployed to chaosnet
Litecoin: Deployed to chaosnet

UTXO Chains

Dogecoin: Complete, will be activated after MCCN
ZCash: Scoped, rain-checked

Cryptonote

Haven: MSS complete, rain-checked until after MCCN stability.
Monero: Pending Haven implementation

Custom

Cardano — Scoping
Polkadot: Demo’d, rain-checked until after MCCN stability.
Avalanche: Scoped, WIP with team to investigate options
Zilliqa: Scoped, rain-checked

EVM Chains

Binance Smart Chain: Likely after MCCN
Ethereum Classic: Rain-checked
Rootstock: Rain-checked
Arbitrum: Rain-checked

IBC

A development partner has been found and will begin building IBC bridges.

Pending IBC integration — Cosmos, Terra, Kava, Secret Network, Injective Protocol, Sifchain, Akash Network — currently scoping with Akash team.

Community Work

The following tools receive regular updates:

THORSwap Client

THORSwap.finance is being maintained by the dev team behind BEPSwap.com. They carry over the same design plus all the UX learnings from BEPSwap.

ASGARDEX Client

V0.1 was released for Chaosnet

ASGARD.EXCHANGE

Weekly Update:
- Input address validation
- Retry transaction polling if request fails due to CORs issue
- Helping with XDEFI code integration + prepping for release
- Sym Deposit Recovery page (https://app.asgard.exchange/deposit-sym-recovery)
- Poll RPC endpoint for THOR native transfers + use Viewblock as explorer for these transactions
- Updates to small UI bugs (ie close button doesn’t close modal)

Vanaheimex Weekly Update:

- approve eth views for swap/deposit
- update breadcrumb structure
- eth send explorer tx fix
- appLock for vanaheimex
- add memo option for send in BNB chain
- scroll bug fix in tx list and pool list
- outbound hash for successful swap
- USD price for eth wallet asset views
- other UI/UX bug fixes

shapeshift.com Weekly Update:

* Thorchain integration complete for both Web and Mobile Apps
* Assets available to swap: BTC, LTC, ETH, USDT, SUSHI
* On Web App, able to do swaps with SS Native Wallet and KeepKey hardware wallet
* Native THOR.RUNE is now supported on SS Native Wallet and KeepKey hardware wallets

Ledger

Work has started on a Ledger App for THORChain

RUNEYield.info Weekly Update: (19/04–25/04)
iOS App

- Finalized main screen design and implementation.
- Drafted Setting page UI to meet minimum requirements for App Store Beta review process.
- Drafted Wallet address management pages for supporting multiple groups of addresses.

THORCHAIN.NETWORK

Weekly Update (4/19–4/25)

cluster-launcher
- Update Kubernetes and dependencies on Linode, Azure & Digitalocean
- Digitalocean troubleshooting

thormon
- Domain and DNS adjustments
- Backend
— Add Healthcheck module (Thor/RPC/Midgard/Bifrost)
— Increase workers/threads to allow more traffic
— Manage API LB and dedicated network nodes
— Tweak microservice arrangements and replicas in kubernetes
- Frontend
— Solvency deployments
— Improve display formattings
— Improve Pause/Resume Feature

thorchain.org Weekly Update:

- Migrating API calls from Midgard v1 to v2
- Story cards + story carousel added to mobile view
- External links updated to the new documents and tool
- Downscaling issues on some tablet models are fixed
- Minor bug fixes

🌈 TRX1’s Weekly Dev Report (19/04–25/04)

Runiverse

• MCCN connection and solving compatibility issues
• Working on TX animations

THORChain Monitoring bot

• Price chart fix for MCCN
• THORChain stats button in the Metrics menu (User number, bond, add, withdraw stats, network security, APYs and so on + 24 hour change)
• A new daily update of THORChain stats at the alert channel
• Summary about THORChain nodes in the Metrics menu (status, IP, version, bond, slash points)
• A new notification on Node churn (what nodes are churning in and out)
• Hot fixes for changing name and keys

GrassRootsCrypto

Weekly update
-Added new logo and branding
-Preping new video scripts; IL protection explained, how to Asymmetrically use Thorchain, Bitcoiners guide to Thorchain and more
-Waiting on caps to be lifted to make interface vids.
-Recording and releasing vids this week!

http://yt.be/GrassRootsCrypto

Telegram Bot

@thornode_testnet_bot

@thornode_chaosnet_bot

Twitter Bot

https://twitter.com/thor_bot

Delphi Defi Dashboard

Upcoming Release for the Dashboard

https://defi.delphidigital.io/thorchain/chaosnet/network

THORChain Network Explorer

Continual Improvements, now with custom sub-domains for each net:

https://thorchain.net/