How To Setup A Custom Ethereum Testnet

Konrad Rotkiewicz
18 June 2020 · 13 min read

Continuing the series on hosting your own testnets, I'll go into details on how to setup a custom Ethereum testnet and why you would want to do that in the first place. If you haven't read the previous posts and you're interested in the topic, you can check out previous blogposts on How to setup custom Bitcoin testnet and How to set up EOS testnet.

Complete example is available on our GitHub.

Ethereum testnets each developer need to know: Ropsten, Rinkeby and others

Ethereum has 5 different testnets, named after subway stations Ropsten, Kovan, Rinkeby, Sokol and Görli. Each of these networks have different rules and consensus protocols - methods for achieving agreement between users on the network on which block should be included in the blockchain. Ropsten uses proof-of-work, which makes it the closest one to mainnet. Miners are required to complete mathematical puzzles, which are then verified by other users on the network. This also makes it the most unstable one. Did I already mention that testnet coins are worthless? There is no financial incentive for people to mine Ropsten coins. Just look at this chart:

Ropsten coins mining - How to set up Ethereum testnet
Ropsten coins mining - How to set up Ethereum testnet

At the time of writing this post, 15 miners were generating testnet coins in the last 24 hours, while the majority of blocks were mined by just 1 miner! PoW networks are only as secure as the computing capacity behind it. It would be really easy to execute the infamous 51% attack.

There are just a handful of miners doing the work, making it susceptible to the infamous 51% attack. At the time of writing this post, just 15 miners were interacting with the network within the last 24 hours. And this is what happened in 2017 - some malicious actors inflated the block gas limit to 9 billion, followed by sending gigantic transactions, crippling the entire network in the process. It was later revived through a soft-fork which placed a hard limit on gas, making this specific attack impossible. It can still be abused, just not in this specific way.

Other networks use proof-of-authority protocols, such as Aura and Clique. You could call them public-permissioned blockchains. In these networks, certain nodes are given permission to participate in the mining process. It's public in a sense that everyone can connect to the network, execute transactions and read the data stored in the ledger, but only a handful of users can mine blocks. Since you can't participate in the mining process, the only way to obtain coins on the network is through a faucet. The fact that there are different protocols, certain networks work only with Parity or Geth. Görli exists for this exact reason. It was specifically designed to be compatible with most clients.

Why should you host your own testnet?

You might wonder, since there are so many choices out there, why should you bother setting up your own then? We chose this approach for similar reasons as with other currencies.

  • Persistence - there is no guarantee that any of the public testnets won't be reset. This is how Ropsten was born, the previous testnet called Morden was simply too hard to use due to all the junk data, since syncing took a long time.
  • Control - our use case requires us to have control over a large sum of funds. If you read my previous posts you probably already know how I feel about faucets. Don't get me wrong, they're great, it just doesn't work for us. Hosting a custom testnet gives us the most control over where the coins are and when new blocks are mined.
  • Storage requirements - participating in any of these networks requires you to sync up with them. We can significantly lower the storage requirements by simply rolling our own and not bothering with the data we don't need.

Tools to setup custom Ethereum testnet

Our setup will consist of three services:

  1. Single Geth node,
  2. Blockscout,
  3. Postgres

We chose the Clique proof-of-authority consensus protocol, since it perfectly fits our use case. This method does not rely on miners being able to solve complex puzzles but instead it gives certain users permission to act as block validators, also called signers. As long as the majority of signers agree that the block is valid, it will be included in the blockchain. They can vote on either approving or kicking out accounts from the group which gives all of them incentive to behave nicely on the network. Blocks are mined with a fixed rate which we can configure, making sure our transactions are confirmed fairly quickly.

Geth node will serve as a signer and provide RPC API for our users. There is a caveat here - this unfortunately means that everyone will be able to interact with the blockchain as our signer. Since this will be an internal testnet, we can assume that our users will not actively try to break things. We’ll also limit the attack surface by not exposing any sensitive APIs such as ones that allow submitting blockchain transactions. Just bear in mind that technically we’re trading security for simplicity here and even though it works for us, it might not suit your use case.

For an explorer app we went with Blockscout. It’s open-source, fairly easy to set up and works well. It uses a PostgreSQL database for storage.

Setting up the Ethereum testnet

We'll be using Docker to set up a custom testnet. Our network will use two accounts. Buffer, which initially holds all the coins and signer which is used by Geth node to participate in the mining process. Once we generate the keys to these accounts, we will create a genesis.json file describing the initial state of our blockchain and bootstrap it using Docker Compose.

Creating the accounts

To create the accounts we'll use vanity-eth.tk tool. It allows us to quickly generate any key pair we want through a simple web interface. You can leave the default settings and simply press the Generate button.

Creating an account to set up Ethereum testnet - Ulam Labs
Creating an account to set up Ethereum testnet

Generate two key pairs and note them down somewhere, we'll need them later. As an example, I'll use these:

Buffer

  • Address: 0x411167FeFecAD12Da17F9063143706C39528aa28
  • Private key: 0x766df34218d5a715018d54789d6383798a1885088d525670802ed8cf152db5b4

Signer

  • Address: 0x0c56352F05De44C9b5BA8bcF9BDEc7e654993339
  • Private key: 0x0d0b4c455973c883bb0fa584f0078178aa90c571a8f1d40f28d2339f4e757dde

Preparing genesis.json

A file which describes the initial state of our testnet is called genesis.json. First block on our blockchain will be based on the contents of this file. Let's go through all the sections and explain what is happening here.

config section

Config section describes core blockchain settings. What's interesting about it is that this data is not a part of the genesis block. Any network identified by a given block can have entirely different set of configuration options.

{
    "config": {
        "chainId": 5555,
        "clique": {
            "period": 60,
            "epoch": 30000
        }
        ...
    },
    ...
}

Let's go through all the options:

  • chainId - identifies the blockchain. This allows Ethereum networks to coexist while also providing a protection against replay attacks, meaning the transaction that is valid on one network, for example Ethereum Mainnet, is not valid on another one such as Ethereum Classic. In our case we can use any value we want. An up-to-date list of known chain IDs in use is available on chainid.network;
  • clique - describes configuration for Clique consensus protocol. It contains two fields:

    • period - how often blocks will be mined (in seconds). We went with 1 minute, since it seems reasonable;
    • epoch - length of an epoch. Epoch is a period in which signers can vote on either adding or kicking out validators from the group. In our example, votes will be summed up after 30000 blocks. If most (>50%) users agreed on a change to the list, it will be executed. Votes are reset at the end of each epoch.

The remaining options tell the client when certain hardforks to the Ethereum networks occur. These are essentially protocol changes which break compatibility and are usually scheduled ahead of time so that all clients on the network have time to adjust accordingly. Once a certain block height is passed, these changes come into effect.

{
    "config": {
        ...
        "homesteadBlock": 0,
        "eip150Block": 0,
        "eip150Hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
        "eip155Block": 0,
        "eip158Block": 0,
        "byzantiumBlock": 0,
        "constantinopleBlock": 0,
        "petersburgBlock": 0,
        "istanbulBlock": 0,
        "muirGlacierBlock": 0,
        ...
    }
}

Not including the field means that our network does not use the fork. We chose to go with zeros, meaning all the protocol changes are already applied. You can use the Geth source code as reference.

extraData

extraData is a field where you can store arbitrary data that will be saved on the blockchain. We'll use it to set the initial state for Clique.

{
    ...
    "extraData": "0x00000000000000000000000000000000000000000000000000000000000000000c56352F05De44C9b5BA8bcF9BDEc7e6549933390000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
    ...
}

Data is expected to be in a specific format:

overview #max500#
ExtraData - How to set up Ethereum testnet?

We can split it into three parts:

  • First 32 bytes are called EXTRA_VANITY bytes. This section can contain arbitrary data which won't be used for any purpose but it will be stored forever on the blockchain. We chose to leave it as zeros.
  • Second part is an initial list of validators. This section should contain a concatenated list of addresses that are allowed to mine blocks on our network, without the 0x prefix. You can include as many addresses as you want, but each address should be 20 bytes long. Our validator address is 0c56352F05De44C9b5BA8bcF9BDEc7e654993339, so we just pasted it in there.
  • Last part is a proposer seal. It identifies a node which proposes a change to the validator list. Since this is the initial state that everyone on the network agrees on, we should just leave it as zeros.

alloc section

This section describes initial balances of accounts on the network. We chose to allocate all the coins to our buffer account.

{
    ...
    "alloc": {
        "411167FeFecAD12Da17F9063143706C39528aa28": {
            "balance": "0x200000000000000000000000000000000000000000000000000000000000000"
        }
    },
    ...
}

Other options

Remaining options describe our genesis block. These are pretty much self-explanatory.

{
    ...
    "nonce": "0x0",
    "timestamp": "0x5eaa7b09",
    "gasLimit": "0x47b760",
    "difficulty": "0x1",
    "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "coinbase": "0x0000000000000000000000000000000000000000",
    "number": "0x0",
    "gasUsed": "0x0",
    "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000"
}

Our genesis block will be mined on 30.04.2020 at 7:15 UTC as described by the timestamp field. Nonce will be null, since we set it to 0 and we're dealing with a proof-of-authority chain. Initial difficulty is 1. It doesn't really matter in our case since it will be constant for any other block and it'll be calibrated automatically so that blocks are mined with a fixed rate. All rewards collected from the successful mining of this block will be sent to the address set in the coinbase field.

Save your complete genesis.json file in your work directory.

Bootstrapping the network

To bootstrap our network we'll use Docker. Create a docker-compose.yml file. Let's start with the Geth node.

version: '3'

services:
  geth:
    image: ulamlabs/geth-poa-testnet:latest
    environment:
      - ETH_PASSWORD=QfdxTYxkwASj
      - ETH_PRIVATE_KEY=0d0b4c455973c883bb0fa584f0078178aa90c571a8f1d40f28d2339f4e757dde
      - ETH_ADDRESS=0c56352F05De44C9b5BA8bcF9BDEc7e654993339
    volumes:
        - ./genesis.json:/app/genesis.json
    ports:
      - 8178:8178
      - 8546:8546

  ...

Geth node uses our geth-poa-testnet Docker image. We need to provide it a couple environment variables:

  • ETH_PASSWORD - password for keystore file which will be generated on first run. It doesn't really have to be that secure since our private key is already available as an environment variable in our container anyway. It can't be changed without removing containers data;
  • ETH_PRIVATE_KEY - private key for our signer account;
  • ETH_ADDRESS - address for our signer account.

Ports section describes ports that are exposed to our local machine. We went with 8178 for HTTP RPC because MyEtherWallet.com Content Security Policy header prohibits connecting to localhost on any other port. You can use a different port and run MEW offline but I didn't want to make things harder than they need to be. For WebSocket API we kept the default port, which is 8546.

  ...
  postgres:
    image: postgres:12
    command: postgres -c 'max_connections=500'
    environment:
      - POSTGRES_HOST_AUTH_METHOD=trust
  ...

I chose to not expose Postgres to our local machine since it doesn't have to be accessed by anything else than Blockscout. This way it shouldn't conflict with any other Postgres instance which might be running on your machine. Blockscout tends to make a lot of connections to the database, so I bumped up the limit to 500, as the default one is not enough. POSTGRES_HOST_AUTH_METHOD environment variable tells Postgres that it is okay for us to connect to it without a password. It's not secure so please don't do that in production!

  ...
  blockscout:
    image: ulamlabs/blockscout:latest
    links:
      - geth
      - postgres
    ports:
      - 4000:4000
    environment:
      - DATABASE_URL=postgresql://postgres:@postgres:5432/postgres?ssl=false
      - ETHEREUM_JSONRPC_VARIANT=geth
      - ETHEREUM_JSONRPC_HTTP_URL=http://geth:8178
      - ETHEREUM_JSONRPC_WS_URL=ws://geth:8546
      - MIX_ENV=prod
      - BLOCKSCOUT_HOST=localhost
      - COIN=eth
      - NETWORK=POA
      - SUBNETWORK=Local Testnet

Last section in our docker-compose.yml file is Blockscout. We need to tell it to connect to our Geth node and PostgreSQL database. SUBNETWORK is a name for our testnet, I went with Local Testnet. Explorer will be listening on port 4000.

Once you're done, you can bootstrap the network using Docker Compose.

$ docker-compose up -d

Your testnet should be up and running in a couple of seconds, which you can confirm by navigating to http://localhost:4000.

overview #max600#
How to setup Ethereum testnet - Blockscout

As you can see, new blocks are mined every minute.

Making it rain 💰💸

Now that the network is up, we can configure MyEtherWallet to connect to it and submit our first transaction. Let's navigate to MyEtherWallet.com and use our buffer accounts private key to access the wallet with all the coins. Choose Access My Wallet, then Software and Private key.

overview #max600#
overview #max600#

You should get a prompt asking you for your private key. Paste it in. You should now see your account. By default MEW connects to the Ethereum mainnet as indicated by Network tile on the dashboard. Click on Change and Add Custom Network/Node. You'll be asked to enter the network details:

  • URL: http://localhost
  • Type: CUS - CUSTOM
  • Port: 8178
  • Chain ID: 5555
  • Explorer TX URL: http://localhost:4000/tx/[[txHash]]
  • Explorer Address URL: http://localhost:4000/address/[[address]]

It should look like this:

overview #max400#
How to set up Ethereum testnet - Network

After clicking Save, scroll to the bottom of the list and you should see your local testnet under the Custom Networks section. Clicking on it will make MEW connect to your local Geth node. If everything went well, you should see this screen:

how to set up ethereum testnet mew wallet
How to set up Ethereum testnet - MEW

We are rich!

Final notes

To make things easier for your users, you might want to export MyEtherWallet configuration to a JSON file. This way they won't have to deal with the form from the previous step. This is especially important once you have some ERC20 tokens on your network as users will be required to enter contract address, token symbol and decimals for each token. Providing a JSON file saves them the trouble of configuring everything by themselves, since all they'd have to do is import it.

Related blogposts:

How To Setup A Custom Bitcoin Testnet?

How To Setup EOS Testnet?

Sidechains And Their Applications

Share on
Related posts
Cryptography In Security: What Are Three Types Of It?
BLOCKCHAIN

Cryptography In Security: What Are Three Types Of It?

Cryptography is essential for the security of our digital world. Blockchain technology uses a wide range of cryptographic techniques. To be able to understand it, you need to understand the basics of…
4 min read
Public vs. Private Blockchains [Podcast]
BLOCKCHAIN

Public vs. Private Blockchains [Podcast]

Let's get back to blockchain basics and explain one thing, there are two types of blockchain: some are public and some are private and only accessible to people who are given permission to use them…
1 min read
Have a question?
Feel free to contact us.
Get a free consultation!