Reducing the Pain of Big Numbers, and Introducing W3bNumber.

Web3 numbers are too big!

The size of numbers in web3 is a problem. Smart contracts use very large numbers for common things like currency amounts. On the other hand, Javascript can only safely represent integers between -(253-1) and 253-1. When integrating contracts into applications, the maximum JavaScript value that can safely be represented (9007199254740991) only amounts to ~0.009 ETH. Something extra is needed for handling and displaying smart contract numbers on a UI.

I have built numerous dApps from the ground up since the early days of Ethereum. As a new dApp developer back then, I suffered through numerous ‘bignumber’ challenges (and headaches) before I felt fully comfortable working with them. I would have loved to have read a simple introduction and gotten some advice about the expected challenges I would face. This article aims to provide just that.

The first section will be a bignumber primer for those not familiar with the concept/tools, and the second part details how we at Yield Protocol introduced a new concept ‘W3Number’ to help us display numbers more effectively on our UIs. Lastly, we show a basic React.js example of implementing what we have learned.

If you are new to dApp development, hopefully, you will walk away with a better understanding of bignumbers and how to use them. If you are a more seasoned dApp developer, hopefully, you can use some of the patterns we use at Yield to make your UI code easier to maintain and extend, and ultimately better for your end user.

PART I: Let’s wrangle some big numbers!

The number system used by many blockchain EVMs does not have any notion of decimal points. Developers will work around this by multiplying all decimal numbers by some large factor, according to the precision required, and dividing them again when needed.

Regular cryptocurrency tokens are often recorded with 18 decimals. This results in some seriously large numbers: for example, one hundred ETH is represented by the integer 100 000 000 000 000 000 000 (or 1x10^20, or 100 quintillions) to allow for the 18 decimals of precision. It is reasonable to assume the number could get bigger than that. It is not uncommon for a contract to track the total supply of a particular token, which could be in the trillions. Ultimately, smart contracts record and use numbers up to 2256-1, well above 253-1 which is the maximum representable in JavaScript.

Thankfully, there are a lot of big-number javascript packages that deal with the arithmetic and comparison of these numbers (over 100 of them on npm). The common ones are BN.js, bigNumber.js, and big.js to name a handful. Basic math and comparisons using these large number packages are a breeze:

// BN.js
const BN = require('bn.js');

const a = new BN('31412365782345324344567');
const  b = new BN('101010', 2);

a.add(b)// arithmetic (addition)
a.lte(b)// comparison (less than equal)

Although similar, each package has slightly different implementations. The package used will probably be heavily influenced by the web3 framework you are using. For example, BN.js is often favored by users of web3.js, while others may prefer the ethers.js own ‘BigNumber’ implementation (which is a spinoff of BN.js with a subtle immutability twist). But ultimately, the choice of package is yours.

While these packages efficiently meet the majority of math/representational requirements in most dApps, they aren’t suited for the ‘most meaningful’ display on a UI. Most offer a toString(), or equivalent function, but this simply returns the ‘contract/evm understandable’ integer version of the number. This is not necessarily what the end user expects to see, for example:

// ethers.js
await provider.getBalance("pi.eth");
>> { BigNumber: "36428926959297445147" }

// web3.js
await web3.eth.getBalance("0x52bc44d5378309EE2abF1539BF71dE1b7d7bE3b5");
>> { BN: "36428926959297445147" }

// raw rpc
curl -X POST https://mainnet.infura.io/v3/a7e4b7372f0940f086c867ddd8e2f808 \ -H "Content-Type: application/json" \ --data \ '{ "jsonrpc": "2.0", "method": "eth_getBalance", "params": ["0x0ADfCCa4B2a1132F82488546AcA086D7E24EA324", "latest"], "id": 1 }'

In the above example, the user’s balance is NOT 36.4 quintillion ETH. 1 ETH is expressed as 1e18 WEI (in Ethereum, the smallest indivisible unit is known as a ‘wei’). This is the number that the EVM and contracts deal in. Imagine seeing your balance in WEI, while it is as accurate as it can be, it isn’t very practical from a UI point of view. The problem is that we need to know where to put the decimal point.

ETH as a token always uses 18 decimal places and it is generally straightforward to manage with whichever web3 utility library you choose:

// ethers.js
ethers.utils.formatEther('1000000000000000000');
⇒ 1.0 ( ETH )

// web3.js
web3.utils.fromWei('1000000000000000000');
⇒ 1.0 ( ETH )

Other derivative tokens could possibly have different decimal places. A widely used example is that of USDC which uses only 6 decimal places. Thankfully, the ERC20 token spec allows us to get the number of decimals the particular token uses via an RPC call to the blockchain:

// ethers.js
const tokenContract = new ethers.Contract(address, abi, provider);
const decimals = await tokenContract.decimals();
⇒ some result

// web3.js
const tokenContract = web3.eth.contract(contractABI).at(contractAddress)
const decimal = await tokenContract.decimals()
⇒ some result

// RPC
curl -X POST [rpc url]  \ -H "Content-Type: application/json" \ --data \ '{ "jsonrpc": "2.0", "method": "eth_getBalance", "params": ["0x0ADfCCa4B2a1132F82488546AcA086D7E24EA324", "latest"], "id": 1 } '
⇒ some result

And the web3 utility packages generally support handling the conversions of an arbitrary decimal number too:

// ethers.js
ethers.utils.formatUnits('100000000', 6)
⇒ 1.0

// web3.js
web3.utils.fromWei('100000000', 6);
⇒ 1.0

Just remember that if you have UI inputs, it will be unlikely the user will provide the value as a bignumber, you may need to convert it back to the correct tokens decimal number format before using it in a contract:

// ethers.js

ethers.utils.parseUnits('100', 18)
⇒ { BigNumber: '100000000000000000000'}

// web3.js
web3.utils.toWei('100', 'ether');
⇒ { BN: '100000000000000000000'}

Great, so now that we know the number of decimals we can show the users what they expect to see, and their input is adequately handled.

PART II: We can handle the bignumber, but what do we do with it now?

The previous section covered the general basics of bignumbers, but from this point on things start to get more opinionated. In this part, we describe a few of the big number of challenges we faced at Yield Protocol, and how we went about dealing with them.

First, it may be useful to explain what we do at Yield Protocol and describe some of our design philosophies and principles. Yield Protocol is a decentralized, open-source financial protocol for fixed-rate borrowing and lending. The details of the Protocol are out of the scope of this article but like all decentralized protocols, our main goals include creating easy-to-use front-ends for users to take advantage of the full power of our product while making the code open source and usable by the community.

From a front-end perspective, the ultimate goal is to open-source a maintainable set of UI tools for interacting with the protocol. This would allow external developers to quickly prototype concepts, test out potential integrations, and build out cool new things that we haven’t even thought of yet!

For these reasons (and maintainability of course), we separate concerns as much as possible. Data stores and data logic are separated from the visual UI. We would like all UI components to be essentially ‘hot-swappable’ and, as far as possible, independent of any particular framework.

This introduces a host of development challenges. One major concern related to big-numbers is that UI components must be agnostic of the token they are representing. This is because in the future, a token could be introduced (not even necessarily by us) that has an arbitrary number of decimal points. Or a token could be introduced that has a substantially different value scale. In either case, we would need a way to indicate that the UI should round to a meaningful number of decimals so we don’t lose any information.

To have UI components that make no assumptions about the tokens they are representing we need to provide each UI component with THREE pieces of information: the integer value, the number of decimals it uses, and important but non-obviously, the number of ‘meaningful’ decimal places from a Users value perspective.

While the first two are pretty straightforward, the third piece of information is more subjective and needs some elaboration. We could very easily show all the decimal points on the UI, but it is very distracting and probably won’t look good. Eg. 3.14159265 USDC has a lot of extra info that is probably irrelevant from a user perspective. On the other hand, if we truncate or round the value for all tokens (say to 2 decimal places) we run the risk of losing meaningful information for some tokens. For example, showing 3.14159265 USDC as 3.14 USDC is usually not a big deal to most users because the potential information loss is at most one cent. However, showing 3.141592653589793238462 ETH as 3.14 ETH can matter because the difference in lost precision could represent as much as $20 (even a currently low ETH price 😢) - which might not be acceptable to all users.

At Yield, we work on the assumption that the point at which it becomes meaningful/practical for our users would be around the equivalent of 1 US cent. In other words, showing USDC with 2 decimal places is acceptable, while showing ETH with 6 decimal digits seems to work for us. For future tokens, we may want to show fewer or more decimal places of precision depending on the subjective value to the User.

Ok, so now we have three pieces of information: the integer value, the decimal places of the token, and an estimation of the meaningful decimal places that correspond to roughly 1 cent precision. We package this information for a UI component as an object that incorporates a few extra properties, it looks like this:

interface W3bNumber = {
hStr: string, dsp: number,
big: bigInt
};

hStr (human-readable string): is the complete string version of the number with the decimal point factored in. It shows all the decimals of precision (eg. 3.141592653589793238462 ETH)

dsp (display): is the number that should be displayed. It has the decimal in the correct place AND is rounded to only the relevant/meaningful precision (eg. 3.14 USDC)

The tough decision we took was to ‘pre-process’ the DSP and hStr, and NOT ship the decimal and number of digits to display as separate properties. The logic behind this decision is that it moves potentially troublesome logic away from the UI component level into the data layer. It also adds the possibility for very thin display UIs that don’t need any web3 package when getting the info from an external API.

big: It is also important to include the bignumber itself. We will certainly need to do basic arithmetic and comparisons within the UI component. This obviously won’t be possible as a string, nor accurate as a concatenated value. If you are considering trying to be UI framework independent, we recommend parsing this property as a BigInt.

Whoa, wait. What is BigInt?

BigInt is a numeric primitive recently introduced to Javascript in the recent ES2020. As the name suggests, it is used to natively store and operate large integers outside the safe integer limit. It is still early days in the life of BigInt and the feature set isn’t as complete as other big number packages. However, given that much logic is done in the data layer of our applications, we find that it is complete enough for most of our UI component needs. The reasoning behind using BigInt and not another bignumber package for our W3bNumber is so that the UI is independent of the web3 framework used as well as the bignumber implementation the developer has decided to use.

Some things might help you out when dealing with bigints in the UI component:

1. Keep in mind that you can’t use the strict equality operator to compare a BigInt to a regular number because they are not of the same type -> BUT you can use loose equality in a pinch.

2. If you are using an older JS and if you feel you would like to use BigInt as a separate package you can look at the JSBI library.

3. Like several other wrappers, BigInts aren’t seamlessly serialized in JSON. However, using a replacer parameter is a possible solution to the problem:

const replacerFn = (key, value) =>  {
if (key === 'big') { return value.toString() }
return value;
}

const w3bNumber = {
dsp: 3.14 ,
hStr: "3.141592653589793238462",
big: BigInt('3141592653589793238462')
};

const stringified = JSON.stringify(w3bNumber, replacerFn);

( Just remember to use the reviver parameter when bringing them back in ):

const reviverFn = (key, value) => {
if (key === 'big') { return BigInt(value) }
return value;
}

const payload = '{"display":"3.14","hstring":"3.141592653589793238462","big":"18014398509481982"}';

const parsed = JSON.parse(payload, reviverFn);

PART III: Pulling it all together with a simple React.js example.

Recently, I needed to tack together a react form that seamlessly handled user ETH input (in human-understandable numbers), and then use that input (represented in WEI) to interact with the blockchain at a later stage. Using W3bNumber made this super easy:

export interface W3bNumber {
dsp: number;
hStr: string;
big: BigInt;
}

The app. tsx and visual components looked like this:

// app.tsx
import {InputContext, InputProvider} from './InputContext';

/* this InputForm, can get dropped anywhere in the app */
export const InputForm = () => {
const [input, handleInput] = useContext(InputContext);
return (
<input
name="invest_amount"
type="number"
value={input.hStr}
onChange={(el) => handleInput(el.target.value)}
/>
);
};

export const DisplayInput = () => {
const [input] = useContext(InputContext);
return (
<div> {input.dsp} <div>
);
};

/* Don't forget to wrap the app with the context provider */
export const App = () => {
return (
<InputProvider>
<InputForm />
<DisplayInput />
</InputProvider> );
};

There is nothing fancy at all about InputForm.tsx or DisplayInput.tsx; they are simple vanilla react components. Notice how the components need to know nothing about the blockchain, or the token they are representing. This is all handled in a separate data layer, in this case, a React Context.

The simplified context supporting this component looks something like this:

// InputContext.tsx
Import {ethers} from 'ethers';

/* A basic react context implementation - this separates any blockchain logic from the visual input component */
export const InputContext = createContext<any>({});

export const InputProvider = ({ children }: any) => {

/* Here, the context state is a w3bNumber and set to '0' (in prod use a reducer) */
const [input, setInput] = useState<W3bNumber>({ dsp: 0, hStr: '0', big: 0n });

/* The following function parses the input to W3bNumber and sets it as the new state */
const handleInput = (input: string) => {
const inputAsWei = ethers.utils.parseUnits(input.toString(), 18);
const inputAsBigInt = BigInt(inputAsWei.toString());
const inputAsString = ethers.utils.formatUnits(inputAsWei, 18);

const inputAsNumber = parseFloat(input).toFixed(6);
setInput ({
dsp: inputAsNumber,
hStr: inputAsString,
big: inputAsBigInt,
})
};

/* the context provider with the input state (w3bNumber) and handleInput() function is returned as the value */
return (

<InputContext.Provider value={[input, handleInput]}>

{children}

</InputContext.Provider>

  )

};

The context does the heavy lifting in terms of parsing the input (which arrives as a string) into a w3bNumber. If we were dealing with other tokens, it is here where we would parse them accordingly. All blockchain interactions and utility packages are now effectively separate from the UI components.

Conclusion

While most web3 developers handle bignumbers daily, those coming from other UI development backgrounds may not be too familiar with them. This article served as both an introduction for the unacquainted, as well as an introducing to the W3bNumber, which is what we at Yield are using to wrangle big numbers in our UI components. We ended by showing a simple example of how this could all fit together in a basic React.js app with a clear separation between the display and data layers.

I look forward to hearing about your experiences, and what you find works for you when handling big numbers in your next dApp!