MythX and Continuous Integration (Part 2): Travis

By Dominik Muhs | Tuesday, February 4th, 2020

In the second part of this series on continuous integration, we will build an easy first integration of the MythX API into the Travis continuous integration platform.

Other posts in this series:

This three-part series is about integrating MythX into Continuous Integration systems. In the first part of this series I have shown how to automatically check for smart contract vulnerabilities in CircleCI. In this part we will take a look at another popular open-source CI system: Travis.

Travis CI

Similar to CircleCI, Travis is configured using a YAML file. This time it is located in the project’s root directory as .travis.yml. The basic concepts of Travis are slightly different from other solutions, however. This greatly simplifies build definitions and job descriptions. This will work in our advantage here.

The tl;dr is that Travis has stages. Each stage consists of multiple jobs that are executed in parallel. These jobs can be grouped into builds. One of these builds will be a collection of jobs that do the MythX security analysis steps.

Sounds complicated? Once we dive into the actual configuration it will turn out to be really easy. But do not trust me, read on!

The basic setup

We will again use Python to run the MythX CLI for all our analysis needs. In the previous post we had to use a virtual environment to isolate our dependencies. In Travis we will rely on a dedicated stage that runs in parallel to all the other things we want to do. So we don’t have to isolate our dependencies. Absolute freedom!

Let’s start writing our configuration file and specify the Python version to use:

language: python
python:
  - 3.7

Before the magic starts we need to install the MythX CLI. We do this in the install phase – before any further steps are executed:

install:
- pip3 install mythx-cli

Spoiler: This line is also how you would install the MythX CLI on your machine as well (with an optional --user added into the mix.)

Submitting to MythX

Now for the analysis magic of submitting our smart contracts to MythX and making the build fail in case anything bad is found:

script:
  - mythx --ci --yes analyze --solc-version 0.5.0 contracts/

And that’s it. To dissect the mythx command a bit further:

  • --ci tells the MythX CLI to return 1 if a medium or high severity issue has been found.
  • --yes disables the confirmation prompt asking the user whether they really want to submit the Solidity files.
  • analyze is the subcommand that tells the CLI we want to send an analysis job to the MythX API.
  • --solc-version defines (you guessed it) the Solidity compiler version to compile the given files with.
  • contracts/ is the directory the smart contracts live in. The MythX CLI will walk through this directory recursively and prepare analysis requests for each .sol file that has been found.

By default the CLI submits scans in quick mode. These return after roughly 120 seconds. So our complete Travis CI configuration file is short and sweet:

language: python
python:
  - 3.7

install:
- pip3 install mythx-cli

script:
  - mythx --ci --yes analyze --solc-version 0.5.0 contracts/

Run it!

I have prepared a test repository with a vulnerable smart contract and connected it to Travis with the above configuration. As soon as the CI job is finished, the following output appears:

The CI check has failed and notified me of a vulnerability. Now I want to know exactly how to trigger it so I can write a test case and validate my fix.

The benefit of signing up for an account on the MythX website (adding my API token to the CI’s environment as MYTHX_ACCESS_TOKEN) is that all my CI analyses are directly attached to my account and I can retrieve them later.

In the MythX dashboard I find my submission:

Diving into it, I see the vulnerability overview that was displayed in my CI output again:

Scrolling down, I see that my code has been highlighted with the vulnerability locations:

For each vulnerability I also have the test case data at my fingertips. Nice!

With the click of a button I can copy the raw transaction data and easily bootstrap my Truffle test cases. This helps me to validate the fix in my code and avoid the issue from occurring in the future.

What’s next?

The MythX CLI is continuously improving (CI jokes, hah!). I have written exhaustive documentation and a usage guide, which allows for more advanced patterns such as blacklisting vulnerabilities, opening custom analysis groups, delivering results in various output formats, and much more.

Do you want to store your reports as artifact files attached to the CI job? Just add the -o parameter and configure the path in Travis.

As always, feedback is a gift, and a comment here or directly in the GitHub repository would be greatly appreciated.

Next up, we will generalize our CI approach in part three. Stay tuned!

Share this post:

MythX and Continuous Integration (Part 1): CircleCI

By Dominik Muhs | Tuesday, January 28th, 2020

In the first post of a new series, we discuss integrating security considerations to a continuous integration pipeline, starting with CircleCI.

Conveyor belts. Because continuity. Get it?

Continuous testing of applications can be hard to figure out. While it is difficult to measure CI/CD adoption, the blockchain ecosystem offers a great opportunity to adopt in-depth testing and continuous delivery pipelines where they make sense: right from the start.

At MythX we don’t mind which technologies you are using to get things done. Choosing the right tool for the right job means that service providers must not make assumptions about their client’s technologies. Security testing especially should be widely available and easy to use.

Behind the scenes the MythX team is working on the tech behind the smart contract analysis to make adding security to your workflow as simple as possible. For instance, we continuously run constructed samples against MythX in our QA process to keep our result quality as high as possible. This has several advantages:

  • Avoid breaking changes. If things go wrong we will notice quickly and can iterate until the change is production-ready.
  • Granular insights. Tests run on small change sets, one step at a time. This makes breaking changes obvious and allows for concise fixes.
  • Save time on code reviews. We improve and review our internal test suites regularly. If they pass, confidence in the code’s correctness rises. That in turn keeps reviews focused on the business logic that matters.

Why CI?

There are plenty of reasons to use Continuous Integration (CI), but the above points have a nice side-effect: they also apply to smart contract development, or the software development lifecycle (SDLC) in general.

There are countless CI solutions out there and, as mentioned previously, we don’t mind which ones teams settle on.

That is why in this three-part series we will take a look at two of the most popular Continuous Integration solutions for open-source projects: CircleCI (this post), and Travis. Afterwards, we will generalize our approach to apply to other CI systems. Using the official MythX CLI it is fairly easy to get a basic test setup going.

Let’s jump right in!

CircleCI

CircleCI is configured using a YAML file located in .circleci/config.yml to describe the build steps. It defines workflows, which in turn consist of jobs. Each of the defined jobs contain a number of steps that are run. (CircleCI offers great tutorials if you need a quick refresher.)

In our case, this will be downloading MythX CLI, and running it against a set of given smart contracts. First we define our basic config:

version: 2
jobs:
  build:
    docker:
      - image: circleci/python:3.8
    steps:
      - checkout

This will define a job called build. All the steps inside this job will be executed in a Docker container running Python 3.8. The MythX CLI also runs on Python 3.6+ and PyPy.

Now for the meaty stuff. Let’s open up a virtual environment and install the MythX CLI into it. This makes sure that our Python application does not collide with any other Python code running in CI, and it avoids a range of CircleCI permission issues.

In our first step we checkout the code at the commit the CI job was given.

- run:
  name: Setup virtual environment
  command: python -m virtualenv venv
- run:
  name: Install MythX CLI
  command: |
    . venv/bin/activate
    pip install mythx-cli

There is nothing spectacular here. We define two steps, three commands, and we are set up. Now, submitting a contract to MythX can be done in a single command.

- run:
  name: Analyze with MythX
  command: |
    . venv/bin/activate
    mythx --ci --yes analyze --solc-version 0.5.0 contracts/

Well, actually two commands: Jump into the virtual environment, and then analyze the code.

There are a few things in the mythx call to note:

  • --ci tells the MythX CLI to return 1 if a medium or high severity issue has been found.
  • --yes disables the confirmation prompt asking the user whether they really want to submit X Solidity files.
  • analyze is the subcommand that tells the CLI we want to send an analysis job to the MythX API.
  • --solc-version defines, you guessed it, the Solidity compiler version to compile the given files with.
  • contracts/ is the directory the smart contracts live in. The MythX CLI will walk through this directory recursively and prepare analysis requests for each .sol file that has been found.

By default the CLI submits scans in quick mode. These return after roughly 120 seconds. Now our full CircleCI config looks like this:

version: 2
jobs:
  build:
    docker:
      - image: circleci/python:3.8
    steps:
      - checkout
      - run:
          name: Setup virtual environment
          command: python -m virtualenv venv
      - run:
          name: Install MythX CLI
          command: |
            . venv/bin/activate
            pip install mythx-cli
      - run:
          name: Analyze with MythX
          command: |
            . venv/bin/activate
            mythx --ci --yes analyze --solc-version 0.5.0 contracts/

Run it!

Below I have prepared a test repository with a vulnerable smart contract and connected it to CircleCI with the above configuration. As soon as the CI job is finished, the following output appears:

The CI check has failed and notified me of a vulnerability. Now I want to note exactly how to trigger it so I can write a test case and validate my fix.

So I have signed up for a free account on the MythX website and added my API token to the CI’s environment as MYTHX_ACCESS_TOKEN. That way all my CI analyses are directly attached to my account and I can retrieve them later. In the dashboard’s analysis listing I find my submission:

Diving into it, I see the vulnerability overview that was displayed in my CI output again:

Scrolling down, I see that my code has been highlighted with the vulnerability locations:

For each vulnerability I also have the test case data at my fingertips.

Nice!

With the click of a button I can copy the raw transaction data and easily bootstrap my Truffle test cases. This helps me to validate the fix in my code and avoid the issue from occurring in the future.

What’s next?

The MythX CLI is continuously improving (CI jokes, hah!) but I have written exhaustive documentation and a usage guide.

In addition to what I showed here, MythX CLI allows for more advanced patterns such as blacklisting vulnerabilities, opening custom analysis groups, delivering results in various output formats, and much more.

Do you want to store your reports as artifact files attached to the CI job? Just add the -o parameter and configure the path in CircleCI. The possibilities are endless!

Feedback is a gift and a comment here or directly in the GitHub repository is greatly appreciated. In the next part, we will be talking about the same integration in Travis CI in part two. Stay tuned!

Have you signed up for a free MythX account yet? Get started today and integrate with your own CI pipeline.

Share this post:

A beginner’s guide to MythX

By Mike Pumphrey | Tuesday, November 26th, 2019

A detailed, step-by-step howto guide on how to use MythX with Remix, showing as well as the differences between MythX and MythX Pro.

MythX is a tool for finding smart contract weaknesses. For our single developers and dev teams, we offer two plans: MythX and MythX Pro.

(We also offer custom plans too; contact us for details.)

We recently posted about the differences between MythX and MythX Pro. But you may find it more useful to see an actual scenario involving testing a smart contract using MythX.

Let’s meet Sam.

Continue…
Share this post:
Category: