CS120: Bitcoin for Developers I

Let's start right at the beginning. What is Bitcoin? 

It certainly is software, but it's also more than that. It's a digital currency, but it's not like the digital currencies that came before it. Bitcoin is the first of its kind, a truly decentralized digital currency that uses economics to incentivize a decentralized ecosystem around it. 

Wrapping your head around the Bitcoin ecosystem can be a challenge. We'll begin by diving into what Bitcoin is, and its history so far.

When trying to understand what Bitcoin is and does, it's helpful to start with an understanding of the context in which it was build and the problem it was trying to solve. There were many digital currencies before Bitcoin, but Bitcoin was the first decentralized digital currency. Creating a digital currency without a central authority was the problem that was being solved for. 

Bitcoin was first introduced to the world On October 31, 2008, with the publishing of the Bitcoin white paper Bitcoin: A Peer-to-Peer Electronic Cash System. The paper gives insight into the motivations and architecture of the system. Much of what is covered in the paper are topics that we will dive into in later units. So, we recommend reading through it briefly now and coming back to it often throughout your studies. 

One of the core components of the Bitcoin system is the peer-to-peer network that it runs on. While peer-to-peer, or P2P, networks existed before Bitcoin, understanding what is happening on the Bitcoin P2P network is fundamental to understanding Bitcoin. 

To grasp what is happening on the Bitcoin network, read this chapter that discusses what the computers, or nodes, on the Bitcoin network are doing. It covers node functions such as running a wallet, mining, maintaining a copy of the blockchain, and routing. We'll get more in-depth on wallets and mining in later sections. Here you'll want to focus on the last two functions mentioned, maintaining a record of all transactions made on the network by keeping copies of all blocks in the blockchain and validating and propagating transaction data.

Very understandably, given the name, when people visualize the Bitcoin network they often picture individual coins moving around a network. However, when you dig into the software you'll find that the concept of individual coins does not exist in the Bitcoin program. What does exist is software that helps the network to manage a shared ledger, the blockchain. It's a ledger that keeps a track of the inputs and outputs of transactions. The unit that the software calculates transactions in is what has become known as a "satoshi" which is 0.00000001 of what we generally think of as a Bitcoin. 

This chapter covers transactions and starts by explaining transaction inputs and outputs, and unspent transaction outputs, or UTXOs.

How does the Bitcoin network decide who has what? If no one is in control of the network, how is sharing of the ledger maintained? What happens when nodes on the network disagree about what transactions took place? Wrapping your head around consensus algorithms is one of the trickier parts of understanding how Bitcoin and other cryptocurrencies work. 

This video introduces and walks through the mechanics of consensus. We'll cover some of the technical specifics of consensus (more specifically, the Nakamoto Proof-of-Work consensus) in Unit 7.

If no one is in control of Bitcoin why can't someone take it over? With so much value in the network, there is a huge incentive to do so. Can the developers control Bitcoin? Can the miners control Bitcoin? 

This video covers the 5 consensus constituencies and how it would not be to the advantage of any of them to ignore the others. You'll want to watch the section between 1:13 and 1:16.

It all started with Bitcoin, but since then many more versions of this technology have sprung up. Bitcoin is an open system, but many of the other blockchain-based systems that have come around since Bitcoin's invention are closed systems. 

Why does this distinction matter? What makes a blockchain-based technology an "open blockchain" system? This video covers the features of an open blockchain system such as neutrality and censorship resistance and why they matter. 

Now that you have an idea of what Bitcoin is doing, let's go see it in action! In this quick exercise, we'll learn how to use a blockchain explorer to find the status of a transaction on the network. 

Bitcoin is called a cryptocurrency because of its heavy use of cryptography. Here we'll take a quick look at symmetric and asymmetric encryption, as well as the two cryptographic functions that Bitcoin uses: digital signatures and hashing.

Now that we've covered the basics of public and private keys, let's dive in a bit deeper and take a look at how Bitcoin generates these keys.

Hashing is used extensively in Bitcoin, from data structures to mining, transaction IDs, and more. Hashing algorithms have many properties that make them very useful. But not all hashing algorithms are the same. SHA256, the algorithm that Bitcoin uses, is well suited to Bitcoin's needs. 

This brief video will introduce you to the basics of hashing.

A Bitcoin transaction isn't valid without a signature. Signatures are a big component in preventing fraud on the network. And where do these signatures come from? Yup, private keys! Your private keys give you the ability to transfer your Bitcoin. This short video will introduce you to the basics of cryptographic signatures.

One of the ways that Bitcoin uses cryptographic keys is in generating Bitcoin addresses, which are often derived from public keys. This chapter covers public keys and how they are used to generate addresses.

The security and integrity of Bitcoin depend largely on private keys, and these keys are randomly generated large numbers. This often feels a bit insecure. You may worry that someone may coincidentally generate the same private key as you. It can be a bit difficult to comprehend the magnitude of the numbers being dealt with here. Since this is such an important piece of the Bitcoin system, let's take a few minutes to appreciate the incredible unlikelihood of generating the same private key as someone else on the network. 

While Bitcoin doesn't actually encrypt any data, it relies upon public and private keys pairs. Working with Bitcoin requires familiarity with these keys and the best way to get familiar with technology is to use it! In this exercise, we'll generate a key pair and use those keys to encrypt and decrypt data with a bit of Python.

We have already covered the basics of digital signatures, so it's time to dive into the specifics of the digital signatures used in Bitcoin. This video explains how Elliptic curve cryptography is used to sign data so that the origin of the signature can be validated, but the private key used to create the signature can not be discovered.

Let's put your new knowledge of cryptographic signatures to use by signing a message and validating it! This exercise will walk you through the process, allowing you to get more familiar with what is needed to create and validate a digital signature.

To understand why digital signatures are integral to Bitcoin transactions, you'll need to learn a bit about the structure of Bitcoin transactions. This chapter will introduce you to what's happening "Behind the Scenes" in these transactions. We'll discuss this more in Unit 5.

Let's dig into when, where, and how Bitcoin uses Elliptic curve signatures in transactions. This chapter covers the importance of signatures to transactions, the three purposes these signatures serve, and how they are applied.

This video walks through the locking and unlocking of transactions and how signatures play a role, specifically in a type of Bitcoin transaction called Pay to Public Key Hash or P2PkH. In addition, the video covers when, how, and by whom these unlocking scripts and signatures are validated. Watch from the start to 9:15.

Let's start our deep dive into how Bitcoin uses hashing by refresh our memory on what a hash is and how it works. This video explains the basics of hashing and its utility.

The first hash functions were created in the 1970s and have evolved rapidly since then. Many of the early hash functions have been "broken" or are no longer secure as methods have been discovered to essentially reverse these types of hashes. As the utility of a hash comes largely from the infeasibility of reverse it, using secure hashing algorithms is vital. Here, we'll review what purposes hashing functions can serve and walk through some more popular algorithms. 

Let's see hashing in action. In this exercise, we'll try out hashing data of varying lengths, which will allow us to verify that the outputs are deterministic and uniform in length.

Bitcoin makes heavy use of hashing, from addresses to transactions IDs and mining. This chapter introduces the mining process. Let's start with some background on the mining process by looking at the monetary aspects of mining, incentives, nodes, and transaction validation. Later, we will discuss some of the more technical aspects of mining.

Now that we have some background on what the mining process does for Bitcoin, let's cover the specifics. This chapter covers the technical process, including mining fees, block data, block headers, and Proof-of-Work.

One of the more interesting, and often confusing, aspects of the mining process is the importance of the number of zeros in a block hash and how the mining difficulty is adjusted. This video will help you visualize target hash difficulty and explains how all nodes come to an agreement regarding the hashing difficulty.

Have you ever tried to guess the outcome of a hashing algorithm? Trying out a mining simulator will give you an appreciation for how incredibly unlikely it is to select an input that will give you a specific hash digest. In this exercise, we'll make an attempt at guessing which nonce values will provide us with a desired hash, and have a look at how blocks are linked together.

Before we wrap up this unit, let's take a look at some of the other ways that Bitcoin makes use of hashing. Have you heard of a Merkle tree? This is the data structure that prevents any alteration of transaction data in a Bitcoin block. This page covers the data structure of a block, including Merkle trees.

This section dives into what Bitcoin calls SPV nodes and how they transmit data. Read this section to understand what data is most vital, as well as the trade-offs between security and convenience when it comes to Bitcoin blockchain data.

Bitcoin uses some unique encoding methods, specifically Base58 and Base58Check encoding. This is done largely to prevent errors in transactions. The way that Bitcoin generates addresses, and the formats that they are displayed in, has been given a lot of thought to prevent confusion and errors. This section covers, base58, key formats, various types of Bitcoin addresses, and more.

Bitcoin and Bitcoin interfaces use not only unique encoding methods, but also some fairly esoteric ones. Unless you've been working with cryptography, you probably are not familiar with ASN.1 and DER encoding. However, as these are formats that you are likely to need to deal with when working with Bitcoin, we'll cover these formats, as well as PEM, here briefly.

Hex is a data encoding format that is often used in Bitcoin and Bitcoin applications. When working with this tech, you will often find data displayed in hex format. If you're not already familiar with hex, now is the time, since you'll certainly run into it while working with Bitcoin.

Now, let's take a look at how all this data, that was calculated and transmitted efficiently with the use of a variety of formats, is stored in the blockchain. As you may suspect, efficiency and security are also priorities.

We've covered how data is formatted, transmitted, and stored in Bitcoin. Now, let's see it in action by looking at some real blockchain data.

Let's practice using the encoding methods we've just covered. In this exercise, we'll convert data including keys between binary, DER, hex, and base58.

Why are Bitcoin nodes important? What do they do for the Bitcoin network? This video explains the difference between the roles of miners and the roles of nodes. It also walks through node functionality and why nodes are so important to the Bitcoin network.

And now, the moment we've all been waiting for: let's actually run Bitcoin! We've learned enough, and we're ready! In this exercise, we'll download, install, configure, and run the Bitcoin core implementation.

Bitcoin wallets either are nodes or rely on other nodes to do a variety of things for them, such as broadcast transactions or scan for transactions related to the wallet. The Bitcoin core implementation comes with wallet functionality, which means it performs functions such as key generation, key storage, and constructing and signing transactions. Let's take a closer look at how wallets perform those functions focusing on key generation, storage, and backups.

Now that we are familiar with the functions of nodes and wallets, let's see how they work together by using our Bitcoin regtest node to perform some wallet functions. In this exercise, we'll generate new addresses, have a look at our wallet seed, construct a transaction, sign and broadcast a transaction, and more.

One topic we have yet to cover in detail is transaction fees. They are an important part of any transaction as without them a transaction may get stuck in limbo, known as the mempool, for a long time. Read this section to get familiar with how fees are added to a transaction.

Now we begin our deep dive into Bitcoin scripting! Let's start by reading these sections on scripting.

This chapter covers advanced transactions and scripting. Here, we will learn about different transaction types and the scripts that create them. You'll use concepts like pay-to-script-hash and multi-sig in almost any Bitcoin development project.

Let's finish by learning about conditional clauses and flow control, and taking a look at some examples of complex scripts.

Bitcoin scripting can be hard to read and visualize, and it's very different from what most programmers are used to. This video walks through scripting examples step by step. Watch from the start to 17:50 to get a bit more comfortable with scripting before we try it out in the next exercise. 

Now that we're more familiar with scripting, let's try this exercise to simulate writing real scripts.

Now that we've learned about transactions and scripting, let's review the transaction validation process. This video walks through the checks that each validating node runs on all the transactions that it sees. Watch from the start until 6:37.

Mining is essential to Bitcoin's decentralized consensus process. As we touched on in Unit 1, this process is how we prevent "double spends", how we determine who has what when, and how we decide on the "truth" on the network without any one entity being in charge of the process. This video walks through the mining process in detail. Watch from the beginning through 28:46.

Now, we'll switch gears a bit from mining to the larger process of consensus. Here, we'll cover validating new blocks, the validation process, and blockchain difficulties such as forks.

Mining isn't just about mining rewards and preventing double spends; there are a variety of ways that confusion can occur on a decentralized network. The consensus process is about working through these issues. Let's continue by watching this presentation on mining and consensus and learn more about the technology being used and how the software works through problems such as which chain to follow in a fork. Watch from 28:47 through to 51:07.

To wrap up this section, we'll cover the possible attacks on a blockchain network and how they are mitigated.

Avoiding double spends, confusion, forks, and attacks, aren't the only complexities of a decentralized consensus mechanism, what happens when you want to update the consensus rules? How to you change a protocol that no one controls? In the last part of Chapter 10 we'll read through the various mechanisms for updating the the Bitcoin code and consensus rules. Read the Chapter 10 from "Changing the Consensus Rules" till the end. 

Now that we understand the major elements of consensus, let's learn where to dig up data on these factors and have a look at them.