Debugging Bitcoin Script failures

Developing apps using Bitcoin (or bitcoin related) blockchain is fun. Until you get your first infamous mandatory-script-verify-flag-failed. These come in different flavors, some of them giving reasonable hints, while others being more or less useless:

• Script evaluated without error but finished with a false/empty top stack element
• Script failed on OP_EQUALVERIFY OP
• Non-canonical DER signature

Here you can find all types of errors.

Bitcoin Script

Script is just a regular stack based programming language, like ASM or JVM bytecode. There are number of opcodes you can push to the stack as well as regular values. Script pops opcodes and executes them. If any operation triggers an error, or final result is false (0) then the whole execution is considered a failure:

• 2 2 OP_ADD — Success — Nothing triggers error, final result is 4
• 2 1 OP_SUB — Success — result is 1
• 2 2 OP_SUB — Failure — result is 0
• 2 1 OP_SUB — Success: result is 1
• 2 1 OP_EQUALVERIFY —Failure — OP_EQUALVERIFY triggers an error
• <signature> <public_key> OP_CHECKSIG— SUCCESS — first item (signature) matches second item (public key)

How Script is used

In reality Script is used to create transactions for different address types. The most popular addresses types are P2PKH (Pay to Public Key Hash — the ones starting with “1”) and P2SH (Pay to Script Hash — these start with “3”). P2PKH address is a hash of public key while P2SH address is a hash of a redeem script. The sender is responsible for creating a locking script and attaching it to the output. The receiver on the other hand must attach an unlocking script to input when spending funds. Locking script is appended to unlocking script and such combined script is executed. If and only if the execution succeeds the output can be considered spent. It is important to note that the way locking and unlocking scripts are constructed is determined by received address type

P2PKH

Let’s look at at the output for this transaction which is consumed by the input from this transaction. As we can see it is aP2PKH transaction since receiver’s address starts with “1”. The way a P2PKH transaction is constructed is as follows:

1. For simplicity let’s call sender A and receiver B
2. A detects that B’s address starts with “1” — it isP2PKH a address. It therefore needs to construct an output with locking script compatible with a P2PKH standard
3. A adds following script to the output: DUP HASH160 0x4a0f1e2f2b0d7daeec5265f10bbbbe08ed725e89 EQUALVERIFY CHECKSIG. 0x4a0f1e2f2b0d7daeec5265f10bbbbe08ed725e89 is B’s public key hash. How does A know public key hash of B? Well, in P2PKH address is the same as public key hash base58 encoded. All A had to do was decode58 address. As you can see there is also EQUALVERIFY opcode. This means public key hash will be compared to something. There is also HASH160 opcode. So 0x4a0f1e2f2b0d7daeec5265f10bbbbe08ed725e89 (public key hash) will be compared to hash of “someting” that B must provide.
4. B decides to spend output. B creates input and adds following script to it: 3045022100de3d4ee588bee5c298d998c251a133213925e45f517c95097f603ae1a19a74c302204cd84d713510e010fd1cb0b367cc78f46c98a2c5a15aed728e06d882c365cc3301 0363e1776495149c5a4e92acf453a3caa19000fc804e4ba2024b600ba25e09c943.The first value is B’s signature and the second one is B’s public key.
5. Scripts are concatenated: 3045022100de3d4ee588bee5c298d998c251a133213925e45f517c95097f603ae1a19a74c302204cd84d713510e010fd1cb0b367cc78f46c98a2c5a15aed728e06d882c365cc3301 0363e1776495149c5a4e92acf453a3caa19000fc804e4ba2024b600ba25e09c943 DUP HASH160 4a0f1e2f2b0d7daeec5265f10bbbbe08ed725e89 EQUALVERIFY CHECKSIG
6. Execution step by step is as follows (I used btcdeb script debugging tool to visualize current stack and opcodes left to be executed):

[embed]

P2SH

In case of P2SH address is generated from “redeem script”. Redeem script in simplest form is described as m <pubkey_1> <pubkey_n> n OP_CHECKMULTISIG where n is amount of public keys that can be used for signing input and m is the minimum required number of public keys. For example for a multisig address requiring 2 of 3 public keys to be used for spending outputs, redeem script would look like this — 2 <pubkey1> <pubkey2> <pubkey3> 3 OP_CHECKMULTISIG. The redeem script is created by receiver. Once receiver constructs redeem script it hashes, which finally becomes an address. The P2SH output script (locking script) is pretty simple: OP_HASH160 <redeem script hash> OP_EQUAL The corresponding P2SH input script (unlocking script) looks like this: 0 <signature_1> <signature_n> <redeem script as hex value>

Extracting scripts from transactions

Now that we know how Script is executed, let us take a look at failing example. Suppose our program has just created new shiny transaction and it is trying to broadcast it to the network, but receives back mandatory-script-verify-flag-failed (Script failed an OP_EQUALVERIFY operation. When the transaction broadcasting fails with such a broad error the first thing we should do is grab boradcasted tx and decode it. There are many online tools for this task, however I do not recommend using them. The most reliable and simplest way is to just use the API provided by the bitcoin node you are using. This way you can be sure transaction is decoded correctly:

[embed]

Take a look at vin.scriptSig value — this is the unlocking script — first part of the puzzle. As described above unlocking script in P2PKH are just 2 values — signature and public key of receiver: 30450221009f61f453f44e807fdc538ca21710393d34005dd5709dabd8b6b9ccf09ea0b36a0220420d91a33f43d7b979471220e12cd1025975bd2e6c0bf2a14eb65ad89d578104 02de8f92034b9b3c956c1896d23a628537561e29faa772438aac2265c91ede6519 We need the second part which is locking script created by sender. As we know input is connected to output from some other transaction. Beware —don’t pick locking script from output within same transaction. I know it seems obvious but there are so many similar steps it’s really easy to make such a silly mistake. In order to find output corresponding to the input we need to grab txid from vin object and get transaction:

> bitcoin-cli getrawtransaction bd6645c8c24618e43053cdc5fe8ccc9418c1307a55cd7409f9681df2fd8c2e1e

02000000000103989d0d3ed579749dfb8493521e908fca126e945d4f7c9a6ab32b66f98fb697d1000000001716001469f8008d203190237e2a4dffab0a3196a1b3f673feffffff6fa67f745a9e1d3cfa51d6fd4ba2ed5344cff6131464242609269456d10952520000000048473044022020c9d9c5c978cd4c0ee1ef70ba94ee96d27cc4ee45843587edfc9abf5c3b8ed602201f54b7fad1e68ebbeebc25c278efe80e88924604ca076df5f68374288028cad701feffffff07308461f2aa761d87b3ef4218aae5d600a7e17cf0e2ccb33fa76bd7e55e13aa0000000048473044022046c07b361173d647ca7c65f16e7e5a275c4a88058388588532e7cdde9d332a8a022061c0ffed32e580b19e834e9a34a78d0ee92dca7ab612f66e7fb2a0b3b70dc1b301feffffff02005a6202000000001976a914393164d26cc8a065cb4915f46f6346d63ae2607588ac55be11000000000017a9149660a010c552090f7793826329e918548b24475f8702473044022063f1c620b6712a410651a7c256b1dcf70103eb7d3ac4b2aaeb3ca2ce3685ed1902200d80aaab0a10fd994312f2402adc79e6510275b1ad3f87d053f920768865da35012102a3a8980559174a47e6c36e4dde336ac05184a602aca79515ee2c56d3dff3a4cd000034080000

Now that we have origin transaction we need to decode it and investigate output script:

[embed]

Pay attention which output you select — n must match vin.vout in transaction that failed. Again it’s one of the silly mistakes that can lead to wrong debugging conclusions. Since vin.vout in spending transaction is 0 we must select script for output with n=0: OP_DUP OP_HASH160 393164d26cc8a065cb4915f46f6346d63ae26075 OP_EQUALVERIFY OP_CHECKSIG

Debug it just like any code

Now that we have both locking and unlocking script we are ready to append them (pay attention to order):

30450221009f61f453f44e807fdc538ca21710393d34005dd5709dabd8b6b9ccf09ea0b36a0220420d91a33f43d7b979471220e12cd1025975bd2e6c0bf2a14eb65ad89d578104 
02de8f92034b9b3c956c1896d23a628537561e29faa772438aac2265c91ede6519
OP_DUP
OP_HASH160
393164d26cc8a065cb4915f46f6346d63ae26075
OP_EQUALVERIFY OP_CHECKSIG

Once we have complete script we can execute it step by step manually or using debugging tool like btcdeb. Since it is a simple operations we can just execute it manually:

1. Signature is pushed onto the stack (30450221009f61f453f44e807fdc538ca21710393d34005dd5709dabd8b6b9ccf09ea0b36a0220420d91a33f43d7b979471220e12cd1025975bd2e6c0bf2a14eb65ad89d578104)
2. Public Key is pushed onto the stack (02de8f92034b9b3c956c1896d23a628537561e29faa772438aac2265c91ede6519)
3. Public Key is pushed onto the stack again (OP_DUP duplicated top element)
4. Public Key is popped from stack hashed and pushed back (b27f04e510293c3e530e7b7bdf71f59118030e1e)
5. Public Key Hash is pushed onto the stack (393164d26cc8a065cb4915f46f6346d63ae26075)
6. OP_EQUALVERIFY is executed — it compares two top values from stack and fails script if they are different. This part fails!

Let’s actually make a hash of 02de8f92034b9b3c956c1896d23a628537561e29faa772438aac2265c91ede6519. For that we can use online tools but beware of them. To be honest bitcoin tooling available online are a little messy. When using any tool for the first time validate it against correct outputs. If the tool provides matching results it is safe to use for debugging invalid scripts. The hash is b27f04e510293c3e530e7b7bdf71f59118030e1e which is definitely not 393164d26cc8a065cb4915f46f6346d63ae26075. We were able to find the issue — the public key provided by receiver does not match the public key hash provided by sender! Moving back to the code that created transaction we found out it was signing inputs with wrong key —quick patch was applied and problem solved.

Some other errors

Sometimes there are problems you might be able to spot right away by looking at the script. These might be missing opcode or some particular value (for example any P2SH locking script must start with 0). There are however more difficult cases where error gives you almost no insight into what is going on. Some of them are:

• Providing bad signature for public key in unlocking script for P2PKH will result in very broad Script evaluated without error but finished with a false/empty top stack element. Why? Let’s look how the P2PKH script looks: <receiver_signature> <receiver_public_key> OP_DUP OP_HASH160 <receiver_public_key_hash> OP_EQUALVERIFY OP_CHECKSIG As you can see, after verifying public key and public key hash matches, signature is checked against public key. OP_CHECKSIG however doesn’t trigger error (unlike OP_EQUALVERIFY). It it just returns 0 or 1. In case of invalid signature the stack is left with 0 element which is false thus causing script to fail.
• Different order of public keys in redeem script for P2SH will also result in very general Script evaluated without error but finished with a false/empty top stack element. As you can see comparing redeem script hashes is done using OP_EQUAL which doesn’t trigger error. It can happen if you used the same public keys for creating redeem script during address generation and during input script creation but with different order. When checking signatures order of public keys shouldn’t matter, but unfortunately the redeem script hash is different.

Summary

If you are developing application which creates and broadcasts bitcoin transactions, it is very likely sooner or later you will need to deal with Script errors. Libraries which we use to construct transactions can prevent most of them. Unfortunately still many failures can only be detected when scripts are combined and executed together. We must be aware of it and take extra care when constructing transactions. If, after all, something goes wrong, now you know some tips how to handle it.

Thinking of blockchain adoption? Grab insights from 8 blockchain experts in a free copy of “Blockchain Adoption Kit” ebook.