260 may contain log entries 265 representing transactions which have been committed to the database (i.e. where the transaction manager 230 has provided confirmation that the transaction has been committed to the database to a requesting entity) and those which are not yet committed to the database (i.e. where the transaction manager 230 has not yet provided confirmation that the transaction has been committed to the database to the requesting entry). These transactions which have log entries 265 written in the transaction log 260 but have not yet been considered to be committed to the database may be referred to as non-committed or hidden transactions. Since the requesting entity may attempt to resubmit any non-committed transactions (i.e. transactions where the requesting entity has not yet been provided with confirmation that the transaction has been committed), in some examples (as discussed in more detail below), the database system 200 provides a mechanism to prevent such hidden transaction from being restored to the database system 200 when restoring from the transaction log 260.

The first secure counter 220 is used by the database system 200 to maintain a count representing the number of transaction log entries 265 that have been written to transaction log 260 for which the corresponding transactions have been committed to the database. That is to say, the number of transaction log entries 265 in the transaction log 260 for which confirmation that the corresponding transaction has been committed to the database has been provided to the requesting entity. The use of the first secure counter 220 will be described further below.

The log truncation module 235 truncates the transaction log 260 by generating a snapshot 240 of the database system 200 at a particular point in time. The snapshot 240 effectively combines all the transaction log entries 265 occurring before the particular point in time at which the snapshot 240 is taken. In some examples, there are two types of snapshot that may be produced—data snapshots which represent the rows that have been inserted or updated since the previous snapshot and delta snapshots which represent rows which have been delete since the previous snapshot. In such examples, the latest snapshot 240 is considered to comprise all previous data and delta snapshots that have been taken up to and including the most recent ones. The database system 200 can then be restored by using the latest snapshot 240 and any transaction log entries 265 for transactions occurring after the snapshot 240 was taken. This means that transaction log entries 265 occurring before the point in time that the snapshot 240 was taken can be removed from the transaction log 260 (or truncated) without affecting the ability for the database system 200 to be restored. The log truncation module 235 operates to ensure that only transaction log entries 265 relating to transactions which have been committed to the database are included in the snapshot 240. Therefore, when truncating the log 260 (and generating a snapshot 240) the log truncation module 235 identifies a point in the transaction log 260 at which all log entries 265 prior to that point relate to transactions which have been committed. In some examples, the log truncation module 235 identifies the latest point in time within the log at which all log entries 265 prior to that point relate to transactions which have been committed. By excluding log entries 265 relating to transactions which have not been committed, the log truncation module 235 can ensure the integrity of the snapshot, even if the database goes offline before all transactions for which log entries 265 have been written to the log 260 have been committed.

In examples where the database system comprises a log truncation module 235 (i.e. in databases where truncation of the transaction log 260 is supported), there is the possibility of reinserting the truncated log entries back into the transaction log 260, thereby effectively causing those transactions to occur twice when the database system 200 is restored from the transaction log 260 (once due to the effect that the transaction had on the snapshot that was generated and once when the transaction log entry 265 is replayed on the snapshot). Therefore, in examples, a technique is provided for preventing this from happening, as discussed in more detail below.

Additionally, when restoring the database it is necessary to ensure that the latest snapshot 240 is used to restore the database system 200. Otherwise, the state of the database following the restoration will be incorrect. In examples, where data and delta snapshots are produced, this comprises checking that all previously taken data and delta snapshots up to and including the most recent ones are used. To ensure that the latest snapshot is provided the database system 200 maintains a cryptographic hash for each data and delta file. The hash is updated as blocks are added to the file. Once all writes to a checkpoint file have completed, the hash is saved in the system table along with the file name. The database system 200 checks the integrity of all checkpoint file pairs read during recovery by comparing the hash of their contents with the hash in the root file. Next, the database system 200 uses a state-continuity protocol based on Ariadne (see R. Strackx and F. Piessens “Ariadne: A Minimal Approach to State Continuity”, in 25^thUSENIX Security Symposium (USENIX Security 16). Austin, TX: USENIX Association, 2016, pp. 875-892 for more details on this protocol) to save and restore the system table within the root file while guaranteeing integrity, freshness and liveness.

The second secure counter 245 is used by the database system 200 to maintain a count representing the number of transaction log entries 265 which have been truncated by the log truncation module 235. By combining the information provided by the first secure counter 220 and the second secure counter 245, the database system is enabled to determine how may transactions log entries (for committed transactions) should be present in the transaction log. Specifically, the difference between the first counter and the second secure counter 220 represents the number of committed transaction log entries 265 which should be in the transaction log 260. Of course, the second secure counter 245 is omitted in other examples of the database system 200. The function performed by the second secure counter 245 will be described further below.

The third secure counter 250 is used by the database system 200 to represent the number of transaction log entries 265 which have been written to the transaction log 260. In other words, the third secure counter 250 represents the total number of transaction log entries 265 which have been written to the transaction log regardless of whether the associated transactions have been committed to the database. Of course, the third secure counter 250 is omitted in other examples of the database system 200. The function performed by the third secure counter 250 will be described further below.

The fourth secure counter 255, which may be referred to as the epoch counter, is configured to be incremented during restoration of the database system 200 by the restoration module 225 at least when a value of the third secure counter 250 at the time of restoration is greater than a value of the first secure counter 220 at the time of restoration. When the value of the third secure counter 250 is greater than the value of the first secure counter 220 this indicates that hidden (i.e. non-committed) transactions may be present in the transaction log 260. In some examples, when the database is restored, as discussed further below, any hidden transactions included in the transaction log 260 may be ignored. This gives rise to the possibility that those ignored hidden transactions could be re-inserted into the transaction log and be included in a future restore of the database (when the first secure counter has been incremented to a point that its value is greater than the indications 270 of the third secure counter value contained in the previously ignored hidden transactions), thereby affecting the integrity of the database system 200. In some examples, this is prevented through the use of the fourth secure counter 255 and the inclusion in each of the transaction log entries of an indication 270 of the value of the fourth secure counter 255 at the time those log entries 265 are written to the transaction log. Any previously ignored transactions will indicate a value for the fourth secure counter 255 which is less than its current value, allowing them to be detected and ignored in future restores of the database system 200. Of course, the fourth secure counter 255 is omitted in other examples of the database system 200. The function performed by the fourth secure counter 255 will be described further below.

The secure counters, that is to say the first secure counter 220 and, where present, the second secure counter 245, third secure counter 250 and fourth secure counter 255, are all secure in the sense that they are stored in such a way that their values are persisted between reboots of the host computer system 100 on which the database system 200 operates. The secure counters are also secure in the sense that they are difficult for a malicious user of the system to tamper with.

In some examples, the secure counters are monotonic counters. That is to say, they only allow the counter to be incremented, not decremented. This provides an additional barrier to a would be attacker attempting to alter the logs because the secure counter does not provide any function to allow the count to be decremented (as would be required if log entries 265 were to be removed from the log without detection). The insertion of additional new log entries 265 into the transaction log is generally of less concern because existing mechanisms, such as encrypting the log entries 265 or signing a hash of the log entries 265 using a securely stored private key can provide a good level of protection against such attacks.

Where the secure counters are not monotonic, the representations of the different secure counts may be stored relative to the other secure counts. That is to say, instead maintaining an absolute number of transactions as the third secure count, the third secure count could be implicitly provided by summing the value of the first secure count with a counter which explicitly tracks the number hidden (i.e. non-committed) transactions that may also have log entries present in the transaction log 260. The use of a non-monotonic counter in such examples allows the counts to be decremented as well as incremented, enabling this approach to be used. As a further example of this approach, the second secure count could be implicitly provided by maintaining an explicit count of the number of log entries 265 for transactions which have been committed where those log entries have not yet been truncated. In this example, the second secure count is implicitly provided by subtracting the value of this explicit count from the value of the first secure count 240.

In some examples, the secure counters may be provided by features of the trusted execution environment within which parts of the database execute. As an example, a monotonic counter can be implemented using the wear-limited NVRAM available in the management engine. However, in alternative example, the secure counters are provided by distributed monotonic counter service, which will now be described in more detail below in conjunction with FIG. 3. The distributed monotonic counter service is potentially more flexible and efficient than using features of the trusted execution environment since monotonic counters implemented using such features can be slow to access.

FIG. 3 is a schematic diagram of an exemplary distributed monotonic counter service (MCS) 300 with which examples of the database system 200 may communicate to maintain and retrieve the secure counts 220, 245, 250 and 255. The distributed monotonic counter service (MCS) 300 comprises two or more replicated instances 310 which are distributed across more than one fault domain 320.

A fault domain is defined by the scope with which faults have the potential to impact the provision of computational services. In other words, all services operating within a single fault domain 320 can potentially be made unavailable by a single fault within the hardware providing those services. Whereas, services operating across more than one fault domain 320 cannot be made unavailable by a single fault within the hardware providing those services. In some examples, a fault domain may also encompass all computing services within a particular geographical area which may be impacted by a single event (such as an earthquake or fire) within that geographical area. In such examples, the distribution of the replicated instances 310 of the distributed monotonic counter service 300 across more than one fault domain 320 may require some form of geographical separation between computer systems 100 operating the replicated instances 310 (however, in other examples, this needn't be the case). By distributing the monotonic counter service 300 across multiple fault domains 320, the reliability of the monotonic counter service can be very high and in general, is better the greater number of fault domains 320 the instances 310 are spread across.

Each of the monotonic counter service replicated instances 310 stores their own versions of the count for each of the secure counters. The instances 310 then determine what the current value for each of the secure counters is using a consensus (or agreement) protocol between two or more or all of the replicated instances 310. In some examples, a consensus is reached based on a majority of the replicated instances 310 storing a particular value. In some examples, a consensus may be reached if a particular number or proportion of the replicated instances 310 store a particular value. In such examples, it is not necessary for the values stored by all of the replicated instances 310 to be checked if a certain number of replicated instances 310 are already in agreement. Indeed, where some of the replicated instances 310 are rendered unavailable (for example, due to a fault in the fault domain 310 within which they operate), consensus may be reached based on those replicated instances 310 that are still available.

The use of the monotonic counter service 300 to store (or provide) the secure counters further improves the integrity of the counters. This is because, an attacker seeking to alter the secure counts to allow them to tamper with the entries 265 in the transaction log 260 would have to gain access to a number of the monotonic counter service instances 310 before they are able to affect the values of the counters that are provided from the service 300, greatly increasing the amount of effort required. This is due to the fact that the monotonic counter service 300 determines the current values from the countries based on the consensus between multiple replicated instances 310 or the service 300 meaning that simply altering the secure counter values stored in one of the instances is unlikely to affect the value of that counter which is reported by the monotonic counter service 300.

In some examples, the database system 200 is multi-threaded. That is to say, it includes a plurality of threads (which may be referred to a transaction processing threads) which can process multiple transactions in parallel. In these examples, a separate first counter 220 and, where they are present, separate second and third counters 245 and 250 are maintained for each of the threads (or conceptually, the first, second and third counters may maintain separate counts associated with each of the threads. Each of the transaction processing threads may cause transaction log entries 265 to be written to the transaction log 260 as they independently process different transactions. In these examples, the transaction log entries 265 that are written to the transaction log 260 each include an indication of which of the transaction processing threads caused that transaction log entry 265 to be written (i.e. which transaction processing thread processed the transaction associated with that transaction log entry 265). The transactions for each thread can then be restored using the respective counters for that thread.

FIG. 4 is a flow diagram of an exemplary method 400 for writing transaction log entries 265 to the transaction log 260.

The method 400 starts at an optional operation 410, in which the method 400, increments a value increments the third secure count maintained by the third secure counter 250. This counter represents the number of transaction log entries 265 that have been written to the transaction log 260 (regardless of whether the transactions represented by those log entries have been committed to the database). By incrementing the counter first, the third secure count always reflects the maximum number of log entries 265 that should be retrievable from the transaction log. In some cases, fewer log entries may be retrievable if, for example, the host computer system 100 is rebooted before a transaction log entry 265 is actually written to the transaction log 260. In other examples, the third secure count need not be incremented as a first operation in the method 400, although this may introduce the possibility that the third secure counter 250 does not reflect the maximum number of log entries 265 that should be retrievable from the transaction log 260. Of course, in examples where the database system does not comprise a third secure counter 250 this operation may be omitted in which case the method 400 proceeds directly to an operation 420.

At operation 420, in which the method 400 writes a transaction log entry 265 to the transaction log 260. In the exemplary method 400 illustrated in FIG. 4, the transaction log entry 265 includes an indication 270 of the current value of the third secure counter 250. However, in other examples, where the database system does not include a third secure counter 250, the transaction log does not include this indication 270. In other examples, where the database system maintains the fourth secure counter 255 mentioned above, the transaction log entry 265 further includes an indication of the current value of the fourth secure counter 255. In yet further examples, where the database system 200 comprises a plurality of transaction processing threads, the transaction log entry 265 that is written in operation 420 further includes an indication of which transaction processing thread caused the log entry to be written to the transaction log 260. In some examples, writing the transaction log entry 265 to the transaction log 260 may comprise encrypting the log entry 265 using a securely stored encryption key. In examples, where indications of the third and/or fourth secure counters are included in each of the transaction log entries, these may be included in the log entries 265 as part of the authentication data for the log entries 265 using encrypted authentication. After writing the transaction log entry 265 to the transaction log 260, the method proceeds to an operation 430.

At operation 430, the method 400 waits for confirmation from the transaction log 260 (which may involve waiting for confirmation from the underlying storage abstraction layer and/or hardware) that the transaction log entry has been written to the transaction log 260. In other words, the method 400 waits until the transaction is safely stored such that after this point, it can be guaranteed that the transaction represented by the transaction log entry 265 will be included in any restoration of the database that is performed using the transaction log 260. Once the confirmation that the transaction log entry 265 has been written to the transaction log 260 has been received, the method proceeds to an operation 440.

At operation 440, the method 400 increments the first secure count provided by the first secure counter 220. This operation is performed prior to informing an entity which requested the transaction to be performed that the transaction has been committed as part of the transaction commit callback procedure. Therefore, the first secure count is maintained as a representation of the number of transaction log entries 265 which have been written to the transaction log for transaction which have been committed to the database. After incrementing the first secure count (and completing the transaction commit callback procedure to notify requesting entities that the transaction has been committed), the method 400 proceeds to an optional operation 450.

At optional operation 450, the method 400 determines whether there are any further transaction log entries 265 which need to be written to the transaction log 260, in which case, the method 400 re-iterates back to optional operation 410. Otherwise, the method 400 ends. In some examples, optional operation 450 is omitted such that the method 400 simply ends.

FIG. 5 is a schematic diagram illustrating exemplary relationships between the first, second and third secure counts 220, 245 and 250 and transaction entries 270 within the transaction log 260. In this illustration, the transaction log 260 comprises twenty one transaction log entries 265 labelled T1 to T21.

In this example, the first nine transaction log entries 265 labelled T1 to T9 have been truncated and combined into a snapshot 240. Therefore, when recovering the database system 200 using this exemplary transaction log 260 illustrated in FIG. 5, the twelve transaction log entries labelled T10 to T21 will be provided (since these have not been truncated from the transaction log), and the database system will need to use the snapshot 240 in conjunction with these transaction log entries 265 to recover the database system 200.

Continuing the example, out of the remaining non-truncated log entries, six log entries, labelled T10 to T15 relate to transactions which have been committed to the database, whilst the remaining six non-truncated log entries, labelled T15 to T20 relate to hidden transactions which have yet to be committed to the database.

Therefore, the value of the first secure count 220 in this example is fifteen, because fifteen transaction log entries 265 have been written to the transaction log 260, nine of which have subsequently been truncated into snapshot 240 and six of which have not been truncated.

In those examples that maintain a second secure count 245, the value of the second secure count in this example is nine, because nine of the transaction log entries 265, labelled T1 to T9, have been truncated into snapshot 240. The total number of non-truncated log entries which relate to transactions that have been committed to the database system 200 is therefore obtainable by subtracting the value of the second secure count 245 from the value of the first secure count 220.

In those examples that maintain a third secure count 250, the value of the third secure count in this example is twenty one because a total of twenty one log entries 265 have been written to the transaction log 260. The total number of log entries 265 relating to hidden (or non-committed) transactions is obtainable by subtracting the value of the first secure count 220 from the value of the third secure count 250.

In examples where the transaction log entries 265 include indications 270 of the third secure counter value 250 at the time that they are written, in this example the log entries T1 to T21 may include indications of the values 1 to 21 respectively. Therefore, the truncated log entries can be identified by finding those log entries indicating a value of the third counter which is less than or equal to the value of the second secure count 245 (where a second secure count is maintained). Similarly, the non-truncated log entries which relate to transactions that have been committed to the database system 200 can be identified by finding those log entries indicating a value of the third counter which is greater than or equal the current value of the second secure count 245 (i.e. greater than nine) and less than or equal to the value of the first secure count (i.e. less than or equal to fifteen). Also, the non-truncated log entries which relate to hidden transactions can be identified by finding those log entries indicating a value of the third counter which is greater than the value of the first secure count 220 (and less than or equal to the value of the third secure count 250).

These relationships between the secure counters and the transaction log entries 265 are used by the database system 200 to verify the integrity of the transaction log entries that are used for recovery of the database system 200 and recover the database system 200 in a manner which preserves its integrity, as will now be discussed in conjunction with FIG. 6.

FIG. 6 is a flow diagram of an exemplary method 600 for restoring the database system 200 using the transaction log 260.

At an operation 605, the method 600 receives transaction log entries 265 from the transaction log 260. For example, in the illustration provided in FIG. 5, the transaction log entries 265 labelled T10 to T21 would be received. In some examples, the transaction log entries that are received may comprise a respective indication 270 of a value of the third secure counter 250 at the time the transaction log entry 265 was written (where the database system 200 maintains a third secure counter 250), a respective indication 275 of a value of the fourth secure counter 255 (where the database system 200 maintains a fourth secure counter 255) at the time the transaction log entry 265, and/or an indication of a transaction processing thread which processed the transaction to which the log entry 265 relates (where the database system 200 comprises a plurality of transaction processing threads). After receiving the transaction log entries 265, the method 600 proceeds to an optional operation 610.

At optional operation 610, the method 600 receives a latest (or current) snapshot 240. In some examples, the latest snapshot 240 comprises all previous data and delta snapshots that have been taken up to and including the most recent ones. For example, in the illustration provided in FIG. 5, the snapshot 240 representing the state of the database system after the transactions associated with log entries 265 labelled T1 to T9 had been committed would be received. Once the latest snapshot 240 has been received, the method 600 proceeds to an operation 615. Of course, in some examples, optional operation 610 may be omitted, in which case the method proceeds directly to operation 615.

At an operation 615, the method 600 receives a current value of the first secure count 220. In some examples, this may be received from the distributed monotonic counter service 300. For example, in the illustration provided in FIG. 5, the value of fifteen may be received as the current value for the first secure count 220. The method 600 then proceeds to an optional operation 620.

At optional operation 620, the method 600 receives a current value of the second secure count 245. Again, in some examples, this may be received from the distributed monotonic counter service 300. For example, in the illustration provided in FIG. 5, the value nine may be received as the current value for the second secure count 245. The method 600 then proceeds to an optional operation 625. In examples where the database system 200 does not maintain a second secure count 245, this operation may be omitted, in which case the method 600 proceeds to directly to operation 625.

At an operation 625, the method 600 receives a current value of the third secure count 250. Again, in some examples, this may be received from the distributed monotonic counter service 300. For example, in the illustration provided in FIG. 5, the value twenty one may be received as the current value for the third secure count 250. The method 600 then proceeds to an optional operation 630. In examples where the database system 200 does not maintain a third secure count 250, this operation may be omitted, in which case the method 600 proceeds directly to operation 630.

At an operation 630, the method 600 receives a current value of the fourth secure count 255. Again, in some examples, this may be received from the distributed monotonic counter service 300. The method then proceeds to an operation 635. In examples where the database system 200 does not maintain a fourth secure count 255, this operation may be omitted, in which case the method 600 proceeds directly to operation 635.

Whilst the retrieval of the current values of the secure counts is represented in FIG. 6 as four separate operations, it will be appreciated that, in other examples, the values of the secure counts may be retrieved in a single operation. For example, a single request may be made to the distributed monotonic counter service 300 to retrieve all of the secure counts at once. Equally, in other examples, the order in which the secure counts is retrieved may be changed.

At operation 635, the method 600 verifies the integrity of the transaction log 260 based on the secure counts that have been received. In some examples, the operation 635 simply verifies the transaction log 260 by determining that the correct number of transaction log entries 265 have been received.

In examples where only the first secure count 220 is maintained (and snapshotting is not supported), this can be determined by checking that the number of transaction log entries that has been received is at least equal to, if not greater than, the value of the first secure count 220. In the example illustrated in FIG. 5, such examples would verify the integrity of the transaction log if at least fifteen transaction log entries had been received.

In examples where a second secure count 245 is maintained (and snapshotting is supported), this can be determined by checking that the number of log entries that has been received is at least equal to, if not greater than, the value of the second secure count 245 subtracted from the value of the first secure count. In the example illustrated in FIG. 5, such examples would verify the integrity of the transaction log if at least six transaction log entries had been received.

In examples where a third secure count 250 is maintained, this can be determined by checking that the number of log entries that has been received does not exceed the value of the third secure count 250 (or the value of the second secure count 245 subtracted from the value of the third secure count 250 where the second secure count is also maintained). In the example illustrated in FIG. 5 (which supports snapshotting and maintains the second secure count), such examples would verify the integrity of the transaction log if the number of received transaction log entries does not exceed twelve.

In examples where the transaction log entries 265 include indications 270 of the third secure counter value 250 at the time that they are written, this can be determined by checking that the values of those indications are not less than or equal to the value of the second secure count or greater than the value of the third secure count 250. Other such examples also or alternatively check that the indicated 270 third secure counter values 250 form a complete sequence (i.e. that there are no gaps in the sequence of indicated values which would indicate a missing transaction log entry 265). In the example illustrated in FIG. 5, such examples would verify the integrity of the transaction log 260 if the third secure counter values 250 indicated 270 in each transaction log entry 265 do not include any indications less than or equal to nine or greater than twenty one and that the received indications form a complete sequence (i.e. all the values 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 and 21 are indicated by respective transaction log entries).

In examples where a fourth secure count 255 is maintained and the log entries 265 include indications 275 of the value of the fourth secure count 255 at the time that they are written, this can be determined by checking that the values of those indications are not less than (or are equal to) the current value of the fourth secure count 255.

If the integrity of the received transaction log entries 265 is verified, the method 600 proceeds to an operation 640. Otherwise, the method 600 terminates without restoring the database due to the integrity of the transaction log entries 265 being compromised. In other examples, the method 600 may provide a warning to the user that the integrity of the transaction log entries 265 has been compromised. In yet further examples, the method 600 may permit the user to ignore such warnings and restore the database anyway.

At an operation 640, the method 600 restores the database based on the received transaction log entries 265 and, where a snapshot 240 is received, on the received snapshot 240.

In some examples, the restoration is based on received value of the first secure count 220. In some such examples, the database is restored by replaying the transaction log entries 245 until a number of log entries equal to the first secure count 220 has been replayed.

In other examples, where a second secure count 245 is maintained, the restoration is further based on the value of the second secure count 245. In some such examples, the database is restored by replaying the transaction log entries until a number of log entries equal to the second secure count 245 subtracted from the first secure count 220 has been replayed.

In other examples, where indications 270 of a respective third secure count 250 are included in each of the transaction log entries 265, the restoration is further based on the indications 270 contained in each of the transaction log entries 265. In some such examples, the restoration of the database ignores (or excludes) any transaction log entries 265 for hidden transactions, that is transaction log entries 265 where the indication 270 of the third secure count 250 is greater than the current value of the first secure count 220. In other such examples, the restoration of the database ignores (or excludes) transaction log entries which have been truncated, that is, where the indication 270 of the third secure count 250 is less than the current value of the second secure count 245.

In other examples, where the database system 200 maintains a fourth secure counter 255 and the transaction log entries 265 each include an indication 275 of a value of the fourth secure count 255 at the time the transaction log entry was written, the restoration is further based on the current value of the fourth secure counter 255 and the indications 270 contained in each of the transaction log entries 265. In such examples, the restoration of the database ignores (or excludes) any transaction log entries 265 having an indicated 275 fourth secure count 255 value which is less than the current value of the fourth secure count 255.

At optional operation 645, the method 600 determines whether any of the received transaction log entries related to hidden (i.e. non-committed) transactions. This is determined by comparing the value of the third secure count 250 with that of the first secure count 220. If the third secure count 250 was greater than the first secure count 220 then that indicates that log entries relating to hidden transactions may be present in the received transaction log entries 265. If it is determined that the received transaction log entries 265 may include transaction log entries 265 related to hidden transactions, then the method proceeds to optional operation 650. Of course, in some examples, where a fourth secure count 255 is not maintained by the database system 200, operations 645 and 650 may be omitted, in which case the method 600 simply terminates.

At optional operation 650, the method 600 takes a new snapshot of the entire transaction log 260. This ensures that all of the log entries 265 relating to transactions which have been committed to the database are included in the latest snapshot 240. After the snapshot has been created, the method 900 proceeds to an optional operation 655.

At optional operation 655, the method 600 increments the fourth secure count 255. Incrementing the fourth secure count serves to invalidate all previously written transaction log entries 265, including any hidden transaction that might have been present during restoration of the database. This prevents any hidden transactions from being maliciously reused because, in examples in which the database system 200 maintains the fourth secure count, each transaction log entry 265 includes an indication 275 of the value of the fourth secure count 255 at the time the entry 265 is written to the log. Therefore, the database system 200 can ignore all transactions which indicate a value which is less the current value of the fourth secure count 255.

Alternatively or in addition to the other examples described herein, examples include any combination of the following:

A database system configured such that at least part of the database system is configured to be executed within a trusted execution environment, the database system comprising: a transaction logger configured to write transaction log entries to a transaction log stored outside of the trusted execution environment, each of the transaction log entries representing a transaction processed by the database system; a first secure counter configured to maintain a representation of the number of transaction log entries which have been written to the transaction log for transactions which have been committed to the database; and a restoration module configured to restore the database system based, at least in part, on transaction log entries retrieved from the transaction log and a current value of the first secure counter.

The database system described above further comprising: a log truncation module configured to truncate the transaction log by generating a snapshot representing the state of the database system at a particular point in time and truncating all transaction log entries for transactions occurring before that point in time; and a second secure counter configured to maintain a representation of the number of transaction log entries which have been truncated by the log truncation module, wherein the restoration module is further configured to restore the database system based additionally on a latest snapshot of the database system and a current value of the second secure counter.

The database system described above further comprising: a third secure counter configured to maintain a representation of the number of transaction log entries which have been written to the transaction log, wherein: each transaction log entry written to the transaction log by the transaction logger comprises a respective indication of the value of the third secure counter at the time that the transaction log entry was written; and the restoration module is further configured to restore the database system based additionally on a latest snapshot and the respective values of the third secure counter indicated by each of the transaction log entries.

The database system described above further comprising: a plurality of transaction processing threads, wherein each of the transaction log entries written to the transaction log comprise a respective indication of the transaction processing thread which processed the transactions represented by that transaction log entry, wherein each of the separate transaction processing threads is associated with a separate respective first, second and third secure counter and the restoration module is further configured to restore the database system based additionally on the indication of the transaction processing thread in each transaction log entry and the respective first, second and third secure counters for each transaction processing thread.

The database system described above further comprising: a fourth secure counter configured to be incremented during restoration of the database system by the restoration module at least when a value of the third secure counter at the time of restoration is greater than a value of the first counter at the time of restoration, wherein: each transaction log entry written to the transaction log by the transaction logger comprises a respective indication of the value of the fourth secure counter at the time that the transaction log entry was written; and the restoration module is further configured to restore the database system based additionally on a current value of the fourth secure counter and the respective values of the fourth secure counter indicated by each of the transaction log entries.

The database system described above, wherein the restoration module is further configured to ignore, during restoration of the database system, one or more or all of: transaction log entries which include an indication of a value for the third secure counter which is less than or equal to the current value of the second secure counter; transaction log entries which include an indication of a value for the third secure counter which is greater than the current value of the first secure counter; transaction log entries which include an indication of a value for the fourth secure counter which is less than the current value of the fourth secure counter.

The database system described above, wherein the restoration module is further configured to verify that the integrity of the received transaction log entries has been preserved, by determining that the values of the third secure counter indicated by each of the transaction log entries forms a complete sequence.

The database system described above, wherein the restoration module is further configured to verify that the integrity of the received transactions has been preserved, by: determining that the number of received transaction log entries is not less than the current value of the first secure counter less the current value of the second counter; and determining that the number of received transaction log entries is not greater than the current value of the third secure counter less the current value of the second counter.

The database system described above, wherein the database system is configured to encrypt each of the transaction log entries using a securely stored encryption key.

The database system described above where the secure counters are monotonic counters.

The database system described above, wherein the monotonic counters are provided by a distributed monotonic counter service, the distributed monotonic counter service comprising two or more replicated instances, the two or more replicated instances being distributed across more than one fault domain, wherein the distributed monotonic counter service is configured to determine a current value of each secure count using a consensus protocol between two or more or all of the replicated instances.