When anyone says that a software release contains “breaking changes” it often means that some behavior has changed without a full understanding of the changes. However, even if engineers conduct thorough testing, there’s always a risk that a bug is lurking deep beneath. Getting a handle on these types of bugs is the goal of the simulations test. Below we share a little bit of why we run simulations at Tendermint.
As blockchain protocols grow in complexity due to speed and interchain interaction requirements, we need to ship software that will break compatibility with prior features. Breaking changes are common in software development. However, when we speak of value transfer systems involving cryptocurrency, breaking changes are liable to have impacts unseen until a substantial amount of transactions complete. Since transaction volume increases over time, it’s very difficult to test for these hidden issues with simple integration testing procedures.
An informative simulation test of Stargate requires that we run a version of the protocol for several thousand blocks. Subtle errors that could exist in the decimal number implementation, fee calculation, or state machine transitions can be detected by generating and sending randomized messages. The goal of simulations is to detect failures that could halt a chain and provide as much detail as possible, such as log files and the application state at which a failure occurred.
Source: Cosmos Stargate server simulation logs
In the long term, this can reduce software breakage by running the Cosmos protocol and observing unique issues we may not have predicted. We want to continuously raise the bar for testing all of our code so that it is as exhaustive as possible. Exhaustive testing should lead to faster and more confident upgrades to the Cosmos Hub.
If you are interested in learning more about the simulator and the simulation tools we use, check the Github repository that hosts runsim’s source code, the Cosmos SDK documentation and continue to check our Stargate updates here: https://github.com/cosmosdevs/stargate/.
