Solana: A new architecture for a high
performance blockchain v0.8.13
Anatoly Yakovenko
anatoly@solana.io
Legal Disclaimer Nothing in this White Paper is an offer to sell, or the solicitation of an offer
to buy, any tokens. Solana is publishing this White Paper solely to receive feedback and comments from
the public. If and when Solana offers for sale any tokens (or a Simple Agreement for Future Tokens), it
will do so through definitive offering documents, including a disclosure docum ent an d risk factors. Those
definitive documents also are expected to include an updated version of this White Paper, which may
differ significantly from the current version. If and when Solana makes such an offering in the United
States, the offering likely will be available solely to accredited investors.
Nothing in this White Paper should be treated or read as a guarantee or promise of how Solanas
business or the tokens will develop or of the utility or value of the tokens. This White Paper outlines
current plans, which could change at its discretion, and the success of which will depend on many factors
outside Solanas control, including market-based factors and factors within the data and cryptocurrency
industries, am ong others. Any statements about fut ure events are based solely on Solanas analysis of the
issues described in this White Paper. That analysis may prove to be incorrect.
Abstract
This paper proposes a new blockchain architecture based on P roof
of History (PoH) - a proof for verifying order and passage of time
between events. PoH is used to encode trustless passage of time into
a ledger - an append only data structure. When used alongside a
consensus algorithm such as Proof of Work (PoW) or Proof of Stake
(PoS), PoH can reduce messaging overhead in a Byzantine Fault Tol-
erant replicated state machine, resulting inn sub-second fin al ity times.
This paper also proposes two algorithms that leverage t h e time keep-
ing properties of the PoH l e dge r - a PoS al gor it h m that can recover
from partitions of any s iz e and an efficient streaming Proof of Replica-
tion (PoRep). The combination of PoRep and PoH provides a defense
against forgery of the ledger with respect to tim e (ordering) and stor-
age. The protocol is analyzed on a 1 gbps network, and this paper
shows that throu gh pu t up to 710k transactions per second is possible
with todays hardware.
1
1 Introduct io n
Blockchain is an implementation of a fault tolerant replicated state machine.
Current publicly available blockchain s do not rely on time, or make a weak
assumption about the participants ab i l i t i es to keep time [
4, 5]. Each node in
the network usually relies on their own local clock without knowledge of any
other part i ci p ants clocks in th e network . The lack of a trusted source of time
means that when a message timestamp is used to accept or reject a message,
there is no gu a ra ntee that every other part i ci p a nt in the network will make
the exact same choice. The PoH presented here is designed to create a ledger
with verifi a b l e passage of time, i. e. duration between events and message
ordering. It is anticipated that every node in the network will be able to rely
on the recorded passage of time in the ledger without trust.
2 Outline
The remainder of this article is or g an i z ed as follows. Overall system design is
described in Section
3. In depth descri p t i on of Proo f of History is described
in Sect i on
4. In depth description of the proposed Proof of Stake consensus
algorithm is described in Section
5. In depth descrip ti o n of the proposed
fast Proof of Replication is described in S ect i o n
6. System Architecture and
performance limits are analyzed in Section
7. A high perfor m an ce GPU
friendly smart contracts engine is described in Section
7.5
3 Netwo rk Design
As shown in Figure
1, at any given time a system node is designated as
Leader to generate a Proo f of History sequence, providing the network global
read consistency and a verifiable passage of time. The Leader sequences user
messages and orders them such t h a t they can be efficiently processed by other
nodes in the system, maximizing throughput. It executes the transactions
on the current state that is stored in RAM and p u b l is he s the transactions
and a signature of the final stat e to the replications nodes called Verifiers.
Verifiers execute the same transactions on thei r copies of the state, and p u b -
lish their computed signatures of the state as confirm at i o ns . The pub l i sh ed
confirmations serve as votes for the cons en su s algorithm.
2
Figure 1: Transaction flow throughou t the network.
In a non-partitioned state, at any given time, th ere is one Leader in the
network. Each Verifier node has the same hardware capabilities as a Leader
and can be elected as a Leader, this is done through PoS based elections.
Elections for the proposed PoS alg or i t h m are covered in depth in Section
5.6.
In terms of CAP t h eor em , Con si st en cy is almost always picked over Avail-
ability in an event of a Partition. In case of a large p a rt it i o n , this paper
proposes a mechanism to r ecover control of the network from a partit i o n of
any size. This is covered in depth in Section 5.12.
4 Proof of History
Proof of History is a sequence of computation that can provide a way to
cryptographically verify passage of time between two events. It uses a cryp-
tographically secure function writ t en so that output cannot be p r ed i ct ed
from the input, and must be completely executed to generate the output .
The function is run in a sequence on a si n g l e core, its previous output as the
3
current inpu t , per i odically recording the current out p u t, and how many t i m es
its been called. The output can then b e re-computed and verifi ed by external
computers in parallel by checking each sequence segment on a sep a r at e core.
Data can be timestamped into this sequen ce by appen d i n g th e data (or a
hash of some data) into the state of the function. The recording of the state,
index and data as it was appended into th e sequen ces p rovides a timestamp
that can guarantee that the data was created s om et i m e before the next hash
was generated in the sequen c e. This design also suppo r ts horizontal scaling
as multiple generators can synchronize amongst each other by mixing their
state into each others sequences. Horizontal scaling is discussed in depth in
Section
4.4
4.1 Description
The system is d es i gn ed to work as follows. Wi t h a cryptograph i c hash func-
tion, whose output can n ot be predicted without ru n n i n g the function ( e. g .
sha256, ripemd, etc.), run the fu n ct i on from some random starting value
and take its output a n d pass it as the input into the same function again.
Record the number of times the function has been cal l ed and th e outpu t at
each call. The starting random value chosen could be any string, like the
headline of th e New York times for the day.
For example:
PoH Se qu e n ce
Index Operation Output Hash
1 sha256(”any random startin g val u e” ) hash1
2 sha256(hash1) hash2
3 sha256(hash2) hash3
Where hashN represents the actual h a sh output.
It is only necessary to publish a subset of the hashes and indices at an
interval.
For example:
4
PoH Se qu e n ce
Index Operation Output Hash
1 sha256(”any random startin g val u e” ) hash1
200 sha256(hash199) hash200
300 sha256(hash299) hash300
As long as the hash functi o n chosen i s collision resistant, this set of hashes
can only be co m p u t ed in sequence by a single computer thread. This follows
from the fact that there is no way to predict what the hash value at index
300 is going to be w i th o ut actually running the algorithm from the sta r ti n g
value 300 times. Thus we can thus infer from the data structure that real
time has passed between index 0 and index 300 .
In the example i n Figure 2, hash 62f51643c1 was pr oduced on count
510144806912 and hash c43d862d88 was produced on count 510146904064.
Following the previously discussed properti es of the PoH a l g or i t h m , we can
trust that real time passed between count 51014480691 2 and count 510146904064.
Figure 2: Proof of History sequence
5
4.2 Timestamp for Events
This sequence of ha sh es can also be used to record that some piece of da ta
was created before a particular hash index was generated. Using a ‘combine‘
function to combin e the pi ece of data wit h the curr ent hash at the current
index. The data can simply be a cryptographically un i q u e hash of arbitrary
event data. The combine function can be a simple append of data, or any
operation that is collis i on resistant. The next generated hash re p re sents a
timestamp of the data , because it could have only been generated a ft er that
specific piece of data was inserted.
For example:
PoH Se qu e n ce
Index Operation Output Hash
1 sha256(”any random startin g val u e” ) hash1
200 sha256(hash199) hash200
300 sha256(hash299) hash300
Some external event occurs, like a photograph was taken, or any arbitrary
digital data was cr ea t ed :
PoH Se qu e n ce With Data
Index Operation Output Hash
1 sha256(”any random startin g val u e” ) hash1
200 sha256(hash199) hash200
300 sha256(hash299) hash300
336 sha256(append(hash335, photograph
sha256)) hash336
Hash336 is computed from the appended binary data of hash335 and the
sha256 of t h e photograph. The index and the sha256 of th e photograph are
recorded as part of th e sequence output. So anyo n e verifying this sequence
can then recreate this change to the sequ e n ce. The verifying can still be done
in parallel a n d its discussed in Section 4.3
Because the initial process is still sequential, we can then tell that things
entered into the sequence must have occurred sometime before the fut u r e
hashed value was computed.
6
POH Sequence
Index Operation Output Hash
1 sha256(”any random startin g val u e”) hash1
200 sha256(hash199) hash200
300 sha256(hash299) hash300
336 sha256(append(hash335, photograph1
sha256)) hash336
400 sha256(hash399) hash400
500 sha256(hash499) hash500
600 sha256(append(hash599, photograph2
sha256)) hash600
700 sha256(hash699) hash700
Table 1: PoH S eq u en ce With 2 Events
In the sequence represented by Table
1, photograph2 was created before
hash600, and photograph1 was created before hash336. Insert i n g th e dat a
into the sequence of hashes r esu l t s in a change to all sub seq u ent val u es in the
sequence. As long as the hash function used is collision resistant, and the
data was appended, it should b e computationally impossible to p r e- com p u te
any fu t u r e sequences based on prior knowledge of what data will be inte-
grated into t h e sequence.
The data that is mixed into the sequence can be the raw data itself, or
just a hash of the data with accompanying metadata.
In the examp l e in Figure
3, input cfd40df8... was inserted into the
Proof of Histor y sequence. The count at which it was inserted is 510145855488
and the state at which it was inserted it is 3d039eef3. All the future gen -
erated hashes are modified by this change to the sequence, this change is
indicated by the color change in the figure.
Every node observing this sequence can determine the o r d er at which all
events have been inserted and estimate the real time between the insertions.
7
Figure 3: Inserting data into Proof of History
4.3 Verification
The s eq u en ce can be verified correct by a multicore computer in significantly
less time than it took to generate it.
For example:
Core 1
Index Data Output Hash
200 sha256(hash199) hash200
300 sha256(hash299) hash300
Core 2
Index Data Output Hash
300 sha256(hash299) hash300
400 sha256(hash399) hash400
Given some number of cores, like a modern GPU with 4000 cores, the
verifier can split up the sequence of hashes and their ind ex es into 4000 slices,
and in parallel make sure that each slice is correct from the star ti n g hash to
the la st hash in the slice. If the expected time to produce the seq u en ce is
going to be:
8
Figure 4: Verification using multiple cores
Total number of hashes
Hashes per se con d for 1 core
The expected time to verify that the sequen ce is correct is going to be:
Total number of hashes
(Hashes per second per core * Number of cores available to verify)
In the example in Figure 4, each core i s able to verify each slice of the
sequence in par al l e l . Since al l input strings are recorded i nto the output, with
the counter and state that they are appended to, the verifiers can r ep l i ca t e
each slice in parallel. Th e red colored hashes indicate that the sequence was
modified by a data insertion.
4.4 Horizontal Scaling
Its possible to synchronize multiple Proof of History generators by mix i n g the
sequence stat e from each generator to each other gener at o r, and thus achi eve
horizontal scaling of the Proof of History generator. This scaling is done
without shar d i n g. The output of both gen e ra t or s is n ecess ar y to reconstruct
the full order of events in the system.
9
PoH Gen er a to r A
Index Hash Data
1 hash1a
2 hash2a hash1b
3 hash3a
4 hash4a
PoH Gen er a to r B
Index Hash Data
1 hash1b
2 hash2b hash1a
3 hash3b
4 hash4b
Given generators A and B, A receives a data packet from B (hash1b),
which contains the last stat e from Generator B, an d the last state gen er a to r
B observed from Generato r A. The next sta te ha sh i n Gene ra t or A th en de-
pends on the state from Generator B, so we can derive that hash1b happe ned
sometime before hash3a. This proper ty can be transitive, so if three gen er -
ators are synchronized through a si n g l e common generator A ↔ B ↔ C,
we can trace the depe n d en cy between A and C even though they were not
synchronized directly.
By periodically synchronizing the g en er at o r s, each generator can then
handle a portion of external traffic, thus the overall system can handle a
larger amount of events to track at the cost of true time accur acy due to
network latencies between the generators. A global order can still be achieved
by picking some deterministic function to order any events that are within
the synchronization window, such as by the value of the h a sh itself.
In Figure
5, th e two generators insert each others output state and r eco rd
the operati on . The colo r change indicates that data from the peer had mod-
ified the sequence. The generated hashes that are mixed into each stream
are highlighted in bold.
The synchronization is transitive. A ↔ B ↔ C There is a provable order
of events between A and C through B.
Scaling in this way comes at the cost of availability. 10 × 1 gbps connec-
tions with availability of 0.999 would have 0.999
10
= 0.99 availability.
4.5 Consistency
Users are expected to be able to enforce co n si st en cy of t h e generated se-
quence and make it resistant to attacks by inserting the la st observed output
of the sequence they consider valid into their input.
10
Figure 5: Two generators synchronizing
PoH Se qu e n ce A
Index Data Output Hash
10 hash10a
20 Event1 hash20a
30 Event2 hash30a
40 Event3 hash40a
PoH Hid d e n Sequence B
Index Data Output Hash
10 hash10b
20 Event3 hash20b
30 Event2 hash30b
40 Event1 hash40b
A malicious PoH generato r cou l d p roduce a second hidden sequence with
the events in reverse order, if it has access to all the events at once, or is able
to generate a faster hidden sequence.
To prevent thi s attack, each client-generated Event should contain within
itself the la t est hash that the client observed from what it considers to be
a valid sequence. So when a client creates the ”Event1” data, they should
append the last hash the y have observed.
11
PoH Se qu e n ce A
Index Data Output Hash
10 hash10a
20 Event1 = append(event1 data, hash10a) hash20a
30 Event2 = append(event2 data, hash20a) hash30a
40 Event3 = append(event3 data, hash30a) hash40a
When the sequence is published, Event3 woul d be referencing hash30a,
and if its not in th e sequence prior to this Event, the consumers of the
sequence know that its an invalid sequence. The partial reordering attack
would then be limi t ed to the number of hashes produced while the client h as
observed an event and when the event was entered. Clients shou l d then be
able to write software that does not assume the order is correct for the short
period of hashes between the last observed and inserted hash.
To prevent a mal i ci o u s PoH generator from rewriting the client Event
hashes, the clients can submit a si gn a t u re of the event data and the last
observed hash instead of just the data.
PoH Se qu e n ce A
Index Data Output Hash
10 hash10a
20
Event1 = sign(append(event1 data, h a sh 1 0a ) ,
Client Private Key)
hash20a
30
Event2 = sign(append(event2 data, h a sh 2 0a ) ,
Client Private Key)
hash30a
40
Event3 = sign(append(event3 data, h a sh 3 0a ) ,
Client Private Key)
hash40a
Verificatio n of th i s data r eq u i re s a signa t u re verifi c at i o n , and a lookup of
the hash in the sequence of hashes prior to this one.
Verify:
(Signature, PublicKey, hash30a, event3 data) = Event3
Verify(Signature, PublicKey, Event3)
Lookup(hash30a, PoHSequence)
In Figure
6, the user-supplied inpu t is dependent on hash 0xdeadbeef...
existing in the gen er at ed sequence sometime before its inserted. The blue
12
Figure 6: Input with a back reference.
top l eft arrow indicates that the client is referencing a previously produced
hash. The clients mess ag e is only valid in a sequence that contains the hash
0xdeadbeef.... The red color in the sequence in d i cat es that the sequence
has been m odified by t h e clients data .
4.6 Overhead
4000 hashes p e r second would gener a te an additional 160 kilobytes of data,
and would require access to a GPU with 4000 cores and roughly 0.25-0.75
milliseconds of time to verify.
4.7 Attacks
4.7.1 Reversal
Generating a reverse order would require an attacker to start the malicious
sequence after the second event. This delay should allow any non malicious
peer to peer nodes to communicate about the origi n al order.
13
4.7.2 Speed
Having multiple genera to r s may make deployment more resistant to attacks.
One gen er at o r could be high bandwi d t h , and receive many events to m i x
into its sequence, another generator could be high speed low bandwidth that
periodically mixes with t h e high bandwid t h generator.
The high speed sequence would create a secondary sequence of data that
an attacker would have to reverse.
4.7.3 Long Range Attacks
Long r a n ge attacks involve acquiring old discarded client Private Keys, and
generating a f al si fi e d ledger [
10]. Proof of History provides some protection
against long range attacks. A mal i ci o u s user that gains acce ss to old private
keys would have to recreate a historical record that takes as much t i m e as the
original one they are trying to forge. This would require access to a fast er
processor than the network is currently using, oth er wi se the attacker would
never catch up in hist o ry length.
Additionally, a single source of time al l ows for construction of a simpler
Proof of R ep l i ca t ion (more on that in Secti o n
6). Since the network is de-
signed so that all participants in the network will rel y on a single h i st or i ca l
record of events.
PoRep an d PoH together sho ul d provide a d efen se of both space and ti m e
against a forged ledger.
5 Proof of Stake Consensus
5.1 Description
This specific instance of Proof of Stake is de si gn e d f or quick confir m a t i on of
the current sequence produced by the Proof of History generator, for voting
and selecting t h e next Proof of History generator, and for punish i n g any mis-
behaving validators. This algorithm d epends on messages eventually arriving
to all partic i p at in g nodes wit h i n a certain timeout.
14
5.2 Terminology
bonds Bonds are equivalent to a capital expense in Proof of Work. A miner
buys hardware and electricity, and comm i t s it to a single branch in a
Proof of Work blockchain. A bond is coin that the validator commits
as collateral wh i l e they are validatin g transactions.
slashing The proposed sol u t i on to the nothing at stake problem in Pr oof
of Stake systems [
7]. When a pr oof of voting for a different branch
is published, that branch can destroy the validators bond. This is an
economic incentive designed to discourage validators from confirming
multiple branches.
super majority A super majority i s
2
3
rds of the validators weighted by their
bonds. A super majority vote indicates that the network has reached
consensus, and at least
1
3
rd of the network would have had to vote
maliciously for this br an ch to be invalid. This would put the economic
cost of an attack at
1
3
rd of the market cap of th e coin.
5.3 Bonding
A bon d i n g transaction takes a amount of coin an d moves it to a bonding
account under the users identity. Coins in the bonding account cannot be
spent and have to rem a i n in t h e account until the user removes them. The
user can only remove st ale coins that have time d out. Bonds are vali d after
super majority of the curr ent stakeholders have confirmed the s eq u en ce.
5.4 Voting
It is anticipated that the Proof of History generator will be able to publish
a sign at u r e of the state at a predefined period. Ea ch bonded identity must
confirm that signature by publishing their own signed signature of the st at e.
The vote is a simple yes vote, without a n o .
If super majority of the bonded identities have voted within a timeout,
then this bra n ch would be accepted as valid.
15
5.5 Unbonding
Missing N number of votes marks the coins as stale and no longer eligible for
voting. The user can issue an unbonding transaction to remove them.
N is a dynamic value based on the ratio of stale to active votes. N
increases as the number of stale votes increases. In an event of a large
network partitio n , this allows the larger branch to recover faster then the
smaller branch.
5.6 Elections
Election for a n ew PoH generator occur when the PoH generator fail ur e is
detected. The validator with the largest voting power, or h i gh es t p u b l i c key
address if there is a ti e is picked as the new PoH gener a to r .
A super majority of confirmations are required on the new sequence. If
the new leader fails before a super majority confirmati on s are available, the
next highest validator is selected, and a new set of confirmations is requi r ed .
To switch votes, a validator needs to vote at a higher PoH sequence
counter, and the new vote needs to contain the votes it wants t o switch.
Otherwise the second vote will be slashable. Vote switching is expected to
be designed so that it can only occur at a height that does not have a super
majority.
Once a PoH generator is established, a Secondary can be el ect ed to take
over the transact i on a l processing duties. If a S econ d ar y exists, it will be
considered as th e next leader during a Primary failure.
The platform is designed so tha t the S eco n d ar y becomes Primary an d
lower ran k generators are promoted if an exception is d et ect ed or on a pre-
defined schedule.
5.7 Election Triggers
5.7.1 Forked Proof of History generator
PoH generators are de si gn e d with an identity that signs the generated se-
quence. A fork can o n l y occur in case the PoH generators identity has been
compromised. A fork is detected because two different historical records have
been published on the sa m e PoH identity.
16
5.7.2 Runtime Exceptions
A hardware failure or a b u g, or a intentional error in the PoH generator cou l d
cause it to generate an invalid state and publish a signature of the state that
does not match the lo c al validators result. Validators will publish the correc t
signature via gossip and th i s event would trigger a new round of elections.
Any validators wh o accept an invalid state will have their bon ds slashed.
5.7.3 Network Timeouts
A network timeout would trigger an election.
5.8 Slashing
Slashing occurs when a validator votes two separate s eq u en ces. A proof of
malicious vote wil l r emove the bonded coins from circu l at i o n an d add t h em
to the minin g pool.
A vote that inclu d e s a previous vote on a contending sequence is not
eligible as proof of malicious voting. Instead of slashing the bonds, this vote
removes remove th e currently cast vote on the contending sequence.
Slashing also occurs if a vote is cast for an invalid hash generated by the
PoH generator. The generator is expected to r an d o m l y generate an invalid
state, which would trigger a fa l l b a ck to Secondary.
5.9 Secondary Elections
Secondary and lower ranked Proof of His to r y g en er a t or s ca n be proposed
and approved. A p ro posal is cast on the primary generators sequen ce. The
proposal contains a timeout, if the motion is approved by a super majority
of the vote before the timeou t, the Secondary is considered elected, and will
take over duties as scheduled. Primary can do a soft handover to Secondary
by inserting a message i nto the generated sequence indicating that a hand over
will occur, or inser t i n g an invalid state and forcing the network to fallback
to Secondary.
If a Second a r y is elected, a n d the prim ar y fail s , th e Secon d a ry will be
considered as th e first fallback during an electi on .
17
5.10 Availability
CAP systems that deal with partitions have to pick Consistency or Avail-
ability. Our approach eventually picks Availability, but because we have
an objective measure of time, Consistency is picked with reasonable human
timeouts.
Proof of Stake verifiers lock up some amount of coin in a stake, which
allows them to vote for a particul ar set of tran s act i o n s. Locking up coin is a
transaction that is entered into a PoH stream, just like any other transaction.
To vote, a PoS verifier has to sign the hash of the state, as it was c om p u t ed
after processing all the transact i on s t o a specific position in t h e PoH ledger.
This vote is also entered as a transaction into the PoH stream. Looking at
the PoH ledger, we can then infer how much time passed betwee n each vote,
and if a partition occu rs , for how long each verifier has been unavailable.
To deal with partitions with reasonable human timeframes, we prop os e
a dynamic approach to unstake unavailable verifiers. When the number of
verifiers is high and above
2
3
, the unstaki n g process can be fast. The number
of hashes that must be generated into the ledger is low before the unavailable
verifiers stake is fully unstaked and they are no lon ge r counted for consensus.
When the number of verifiers is below
2
3
rds but above
1
2
, the unstaking timer is
slower, requiring a larg er number of hash e s t o be generated before the missing
verifiers are un st a ked. In a large partition, li ke a partit i o n that is missing
1
2
or more of the verifiers, the unstaking process is very very slow. Transactions
can still be entered into the stream, and verifiers can still vote, but full
2
3
rds
consensus will not be achieved until a very large amount of hashes have been
generated and the unavailable verifiers have been unstaked. The difference
in time for a network to regain liveness allows us as customers of the network
human timeframes to pick a partition that we want to co ntinue using .
5.11 Recovery
In the system we propo se, the ledger can be fully recovered from any failure.
That means, anyone in the world can pick any rand om spot in the ledger and
create a valid fork by appending newly generated hashes and transactions.
If all the verifiers are missing from this fork, it would take a very very long
time for any ad d i t i on a l b on d s to become valid and for this branch to achieve
2
3
rds super majority consensus. So full recovery with zero available validators
would require a very large amount of hashes to be appended to the ledger,
18
and only after al l the unavai l ab l e valid a to r s have been unstaked will any new
bonds be able to validate the ledger.
5.12 Finality
PoH allows verifiers of the network to observe what happened in the past with
some d eg r ee of certainty of the tim e of those events. As the PoH generator is
producing a stream of messages, all the verifi er s are required to submit th ei r
signatures of the state within 500ms. This number can be reduced further
depending o n network conditions. Since each verifica t i on is entered into t h e
stream, everyone in the network can validate that every verifier submitted
their votes within the requ i r ed ti m eo u t wit h o u t actu al l y ob se rv i n g th e voting
directly.
5.13 Attacks
5.13.1 Tragedy of Commons
The PoS verifiers simply confirm the state hash generated by the PoH gen-
erator. There is an econo m i c in ce ntive for them to do no work and simply
approve every g en er a te d state hash. To avoid this c on d i t i o n , the PoH gener-
ator should inject an invalid hash at a random interval. Any voters for this
hash should be sl a sh ed . When the hash is genera t ed , the network should
immediately promote the Secondary elected PoH generator.
Each verifier is require d to respond within a sm a l l timeout - 500ms fo r
example. The timeout should be set low enough that a malicio u s verifier has
a low probability of observing anot h er verifiers vote and getting their votes
into the stream fast enough.
5.13.2 Collusion with the PoH generator
A verifier that is colluding with the PoH generator would know in advance
when the invalid hash is goi n g to be produced and not vote for it. This
scenario is really no different than the PoH identity having a larger verifier
stake. The PoH generator still has to do all the work to produce the state
hash.
19
5.13.3 Censorship
Censorship or denial of service could o ccu r when a
1
3
rd of the bond holders
refuse to validate any sequences with new bonds. The p r ot ocol can defend
against this form of attack by dynam i ca l l y adjusting how fast bonds become
stale. In the event of a denial of service, the larger partit i o n will be designed
to fork and censor the Byzantine bond holders. The la rger network will re-
cover as the Byzantine bonds become stale with time. The smaller Byzantine
partition would not be able to move forward for a longer period of time.
The algori t h m would work as foll ows. A majority of the network would
elect a new Leader. The Leader would then censor the Byzantine bond
holders fro m participating . Proof of History generator would have to continue
generating a sequence, to prove the passage of time, until enough Byzantine
bonds have become stale so the bigger n etwork has a super majority. The rate
at which bonds become stale would be dynamically based o n what percentage
of bonds are active. So the Byzantine minority fork of th e network would have
to wait much longer than the majority fork to recover a super majority. Once
a su per majority has been established, slashing could be used to permanently
punish the By zantine bond holder s.
5.13.4 Long Range Attacks
PoH provides a natural defense against long range attacks. Recovering the
ledger from any point in the past would re qu i r e the attacker to overtake the
valid ledger in t im e by outpacin g the speed of the PoH gen er a t or .
The consensus protocol provides a second layer of defense, as any attack
would have to take longer then th e time it takes to unstake all the valid
validators. It also creates an availability gap in the h i st o r y of the ledger.
When c om p a r i n g two ledgers of the same height, the one with the smallest
maximum partition can be objectively considered as vali d .
5.13.5 ASIC Attacks
Two opportunities for an ASIC attacks exist in t h i s protocol - during parti-
tion, and cheating timeouts in Finality.
For ASIC attacks during Partitions, the Rate at which bonds are unstaked
is non-l i n e ar , and for networks with large partitions the rate i s orders of
magnitude slower then expected gains from an ASIC a t t ack.
20
For ASIC attacks during Fin a l i ty, the vulnerability allows for byzanti n e
validators who have a bon d ed stake wait for confirmations from other nodes
and inject their votes with a collaborating PoH generator. The PoH gener-
ator can then use its faster ASIC to generate 500ms worth of hashes in less
time, and allow for network communicatio n between PoH gener at o r an d t h e
collaborating nod e s. But, if the PoH g en er at o r is also byzantine, there is no
reason why the byzantine generator wouldnt have communicated the exa ct
counter wh e n they expect to insert the fai l u r e. This scenari o is no d i ff er ent
than a PoH gener at o r and all the col l a borators sharing the same identity and
having a single co mbined stake and only using 1 set of h a r d ware.
6 Streaming Proof of Replication
6.1 Description
Filecoin prop o sed a version of Proof of Replication [
6]. The goal of this
version is to have fast and streami n g verifications of Proof of Replication,
which are enabled by keeping track of time in Proof of History generated
sequence. Replication is not used as a consensus algorithm, but is a useful
tool to account for the cost of storing the blockchain history or state at a
high availability.
6.2 Algorithm
As shown in Figure
7 CBC encryption encrypts each block of data in se-
quence, using t h e previously encr yp t ed block to XOR the input da ta .
Each replica t i on identity gen er a t es a key by si g n i n g a hash that has been
generated Proof of History sequence. This ties the key to a replicators iden-
tity, and to a specific Proof of History sequence. Only specific hashes can be
selected. ( S ee Section
6.5 on Hash Selection)
The data set is fully encrypted block by block. Th e n to generate a proof,
the key is used to seed a pseudorandom number generator that selects a
random 32 bytes from each block.
A merkle hash is computed with the sel ect ed PoH hash prepended to the
each slice.
The root is published , along with the key, and the selected hash tha t
was generated. The replication node is requi re d to publish another proof
21
Figure 7: Sequential CB C encryption
Figure 8: Fast Proof of Repl i ca t i on
22
in N hashes as they are generated by Proof of History generator, wh e re N
is approxim a t el y
1
2
the time it takes to encrypt the data. The Proo f of
History g en er a to r will publish specific hashes for Proof of Replication at a
predefined periods. The replicator node must select the next published hash
for genera t i n g the proof. Again, the hash is signed, and random slices are
selected from the blocks t o create the merkle root.
After a period of N proofs, the data is re-encrypted with a new CBC key.
6.3 Verification
With N co r es, each core can stream encryption for each identity. Total space
required is 2blocks ∗ Ncores, sin ce the previous encr y p t ed block is necessary
to generat e the next one. Each core can t h en be used to generate all the
proofs that derived from the current encrypted block.
Total time to verify proofs is expecte d to be equal to the time it takes to
encrypt. The pr oofs themselves consume few random bytes from the b l ock,
so the amount of data to hash is significantly lower then the encrypted block
size. The numb er of replication identities that can be verified at the same
time is equal to the number of available cores. Modern GPUs have 3500+
cores available to them, albeit at
1
2
-
1
3
the clock speed of a CPU.
6.4 Key Rotation
Without key rotation the same encrypted rep l i ca t i on can generate cheap
proofs for mult i p l e Proof of History sequences. Keys are rotated periodically
and each replication is re-encrypted with a new key that i s tied to a unique
Proof of Hist o r y sequence.
Rotation needs to be sl ow enough that its practical to verify replication
proofs on GPU hardware, wh i ch is slower per core than CPUs.
6.5 Hash Selection
Proof of History generator publi sh es a hash to be used by the entire network
for encrypting Pro o fs of Replication, and for using as the pseudorandom
number generator for byte selection in fast proofs.
Hash is published at a periodic counter that is rou gh l y equal to
1
2
the time
it takes to encrypt the data set. Each repl i c at i o n identity must use the same
23
hash, and use the signed result of the hash as the seed for byte selection, or
the encryption key.
The period that each replicator must provide a proof must be smaller than
the encryption time. Other wi se the replicator can stream the encryption and
delete it for each proof.
A malicious generator could inject d a t a into the sequence pri or to thi s
hash to generat e a specific hash. This attack is discussed more in
5.13.2.
6.6 Proof Validation
The Proof of History node is not expected to valid a t e the submitted Proof
of Replication proofs. It is expected to keep track of number of pending and
verified proofs submitted by the replicators identity. A proof is expected to
be verified when the replica t or is able t o sign the proof by a s u per majority
of the validators in the network.
The verifications are collected by the replicator via p 2 p gossip network,
and submitted as one packet that contains a super majority of the validat or s
in the network. This packet verifies all the proofs prior to a specific hash gen-
erated by the Proof of History sequence, and can contain multiple replicator
identities at once.
6.7 Attacks
6.7.1 Spam
A malicious user coul d create many replicator identities and spam the net-
work with bad proofs. To facilitate faster verification, nodes are required
to provide the encrypted data and the entire mer k l e tree to the rest of the
network when they request verificati o n .
The Proof of Replication that is designed in this paper allows for cheap
verification of any additional proofs, a s they take no additional space. But
each identi ty would consume 1 core of encryption time. The replication target
should be set to a maximum si ze of readily available cores. Modern GPUs
ship with 3500 + cores.
6.7.2 Partial Erasure
A repli ca tor node could attempt to partially erase some of the data to avoid
storing the entire state. The number of proofs and the randomness of the
24
seed should make this attack d i ffi cu l t .
For example, a user storing 1 terabyte of data erases a single byte from
each 1 megabyte block. A single proof that samples 1 byte out o f every
megabyte would have a likelihood of collision w i th any erased byte 1 − (1 −
1/1, 000, 0000)
1,000,000
= 0.63. After 5 proofs the likelihood is 0.99.
6.7.3 Collusion with PoH g ener a to r
The signed hash is expected to be used to seed the sample. If a replicator
could select a specific hash in advance then the rep l i ca t or could eras e all
bytes that are not going to be sampled.
A replicator identi ty that is colluding with the Proof of History genera to r
could inject a speci fi c transaction at the end of the sequence before the pre-
defined has h for random byte selection is generated. With e n ou g h cores, an
attacker could generate a hash that i s preferable to the replicators identity.
This attack cou l d only benefit a single replicator identity. Since all the
identities have to use the same exact hash that is crypt og r ap h i c al l y signed
with ECDSA (or equivalent), the resulting signature is unique for each repli-
cator identity, and collision resistant. A single replicator identity would only
have marginal gains.
6.7.4 Denial of Service
The cost o f adding an additional replicator identity is expect ed to be equal
to the cost of storage. The cost of adding extra computati on a l cap a ci ty to
verify all the replicator identities is expected to be equal to the cost of a CP U
or GPU core per replication identity.
This creates an opportunity for a denial of service attack on the network
by c re at i n g a large number of valid replicator i d entities.
To li m i t this attack, the consensus protocol chosen for the network can
select a replication target, and award the replication proofs t h a t meet the de-
sired characteristics, like availability on the network, bandwidth, geolocation
etc...
6.7.5 Tragedy of Commons
The PoS verifiers could simply confirm PoRep without doing any work. The
economic incentives should be lined up with the PoS verifiers to do work,
25
Figure 9: System Architecture
like by splitting the mining payout between the PoS verifier s and the PoRep
replication nodes.
To further avoid this scenar i o , the PoRep veri fi er s can submit false proofs
a small percentage of the t i m e. They can prove the proof is false by providi n g
the function that generat ed the false data. Any PoS verifier that confirmed
a false proof woul d be slashed .
7 System Architecture
7.1 Components
7.1.1 Leader, Proof of His to ry generator
The Leader is an elected Pr oof of History genera t or . It co n su m es arbitrary
user transactions and outputs a Proof of Hi st or y sequence of al l the transac-
tions that guarantees a unique global order in the system. After each batch
of transactions the Leader outputs a sig n at u r e of t h e st a t e that is the result
of running the transactions in that order. This signature is signed with the
identity of the Leader.
26
7.1.2 State
A naive hash table indexed by the users address. Each cell contains the full
users address and the memory required for this computation. For example
Transactio n table contains:
0 31 63 95 127 159 191 223 255
Ripemd of Users Public Key
Account
unused
For a total of 32 bytes.
Proof of St a ke bonds table contains:
0 31 63 95 127 159 191 223 255
Ripemd of Users Public Key
Bond
Last Vote
unused
For a total of 64 bytes.
7.1.3 Verifier, State Replication
The Verifier nodes replicate the blockchain state and provide high availability
of the blockchain state. The replication target is selected by the consensus
algorithm, and the validators in the consensus algorithm select and vote the
Proof of Replication nodes they approve of based on off-chain defined criteri a .
The network could be configured with a minimum P roof of Stake bond
size, and a requirement for a single replicator identity per bond.
7.1.4 Validators
These nodes are consuming bandwid t h from Verifiers. They are vi r tu a l nodes,
and can run on the same ma chines a s the Verifier s or the Leader, or on
separate machines that are specific to the consensus algorithm configured for
this network.
7.2 Network L im it s
Leader is expected to be able to take incoming user packets, orders them
the most efficient way possibl e, and sequences them into a Proof of Hist o r y
sequence that is published to downstr ea m Verifiers. Efficiency is ba sed on
27
Figure 10: Generator network limits
memory access patterns of the transactions, so the transactions are order ed
to minimize fa u lt s and to maximize prefetching.
Incoming packet format:
0 31 63 95 127 159 191 223 255
Last Valid Hash
Counter
u s
Fee
From
Signed
Signature 1 of 2
Signature 2 of 2
Size 20 + 8 + 16 + 8 + 32 + 3232 = 1 4 8 bytes.
28
The minimal payload that can be supported would be 1 destination ac-
count. With payload:
0 31 63 95 127 159 191 223 255
Last Valid Hash
Counter
u s
To Amount
Counter Fee
From
Signed
Signature 1 of 2
Signature 2 of 2
With payload the minimum size: 176 bytes
The Proof of History sequence packet contains the current hash, counter,
and the hash of all the new messages added to t h e PoH sequence and the
state signature after processing all the m essa ge s. This packet is sent o n ce
every N messages are broadcast.
Proof of Hist o r y packet:
0 31 63 95 127 159 191 223 255
Current Hash
Counter
Messages Hash
State Hash
Signed
Signature 1 of 2
Signature 2 of 2
Minimum size of th e output packet is: 132 bytes
On a 1gbps network connection the max i mum number of transactio n s
29
possible is 1 gigabit per second / 176 bytes = 710k tps max. S om e loss 1−4%
is expected due to Ethernet fr am i n g . The spare capacity over the target
amount for the network can be used to in c re ase availability by coding the
output with Reed-Solomon cod es and striping it to the available downstream
Verifiers.
7.3 Computational Limits
Each transaction requires a di ges t verification. This op er a t i on does not use
any memory outside of the transaction message itself and can be parallel i zed
independently. Thus throug h p u t is expected to be limited by the number of
cores available on the system.
GPU based ECDSA verification servers have had experimental re su l t s of
900k operations per second [
9].
7.4 Memory Limits
A naive implementation of the state as a 50% ful l hashtable wit h 32 byte en-
tries for each account, would theoretically fit 10 billion accounts into 640GB.
Steady state random access to this tab le i s measured at 1.1 ∗ 10
7
writes or
reads per second. Based on 2 reads and two writes per transaction, memory
throughput can handle 2.75m transactions per second. This was measured
on an Amazon Web Services 1TB x1.16xlarge instance.
7.5 High Performance Smart Contracts
Smart contracts are a generalized form of t ra n sa ct i on s . These are programs
that run on each node and modify the state. This design leverages extended
Berkeley Packet Filter bytecode as fast and easy to analyze a n d JIT bytecode
as the smart contrac ts language.
One of its main advantages is a zero cost Foreign Function Interface.
Intrinsics, or functions that are implemented on the platform directly, are
callable by programs. Calling the intrinsics suspends that program and
schedules the intrinsic on a high per fo r man ce server. Intrinsics are batched
together to execut e in parallel on the GPU.
In the above examp l e, two different user progra m s call the same intrinsic.
Each program is suspended unti l the b at ch execution of the intrinsics is
30
Figure 11: Executing BPF programs.
complete. An example intrinsic is ECDSA verificati on . Batching these calls
to execute on the GPU can increase through p u t by thousands of times.
This trampoline requires no native operating system thread context swi t ches,
since the BPF bytecode has a well defined context for all the memory that
it is using.
eBPF backend has been included in LLVM since 2015, so any LLVM
frontend language can be used to write smart contracts. Its been in the
Linux kernel since 2015, and the first iterations of the bytecode have been
around si n c e 1992. A single pass can check eBPF for correctness, ascertain
its runtime a n d memory requi r em ents and convert it to x86 instructions.
References
[1] Liskov, Practical use of Clocks
http://www.dainf.cefetpr.br/ tacla/SDII/PracticalUseOfClocks.pdf
[2] Google Spanner TrueTime consistency
https://cloud.google.com/spanner/docs/true-time-external-consistency
[3] Solvin g Agreement with Ordering Oracl es
http://www.inf.usi.ch/faculty/pedone/Paper/2002/2002EDCCb.pdf
31
[4] Tendermint: Consensus without Mi n i n g
https://tendermint.com/static/docs/tendermint.pdf
[5] Hedera: A Governing Council & Public Hashgraph Network
https://s3.amazonaws.com/hedera-hashgraph/hh-whitepaper-v1.0-180313.pdf
[6] Filecoin , proof of replication,
https://filecoin.io/proof-of-replication.pdf
[7] Slasher, A punative Proof of Stake algorithm
https://blog.ethereum.org/2014/01/15/slasher-a-punitive-proof-of-stake-algorithm/
[8] BitShar es Delegated Proof of Stake
https://github.com/BitShares/bitshares/wiki/Delegated-Proof-of-Stake
[9] An Efficient Ellipti c Curve Crypto gr a p hy Signature Server With GPU
Acceleration
http://ieeexplore.ieee.org/document/7555336/
[10] Casper the Friendly Finality Gadget
https://arxiv.org/pdf/1710.09437.pdf
32
