-
Notifications
You must be signed in to change notification settings - Fork 378
/
trillian_log_api.proto
363 lines (321 loc) · 14.6 KB
/
trillian_log_api.proto
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
// Copyright 2016 Google LLC. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
syntax = "proto3";
package trillian;
option go_package = "github.com/google/trillian";
option java_multiple_files = true;
option java_outer_classname = "TrillianLogApiProto";
option java_package = "com.google.trillian.proto";
import "google/protobuf/timestamp.proto";
import "google/rpc/status.proto";
import "trillian.proto";
// The TrillianLog service provides access to an append-only Log data structure
// as described in the [Verifiable Data
// Structures](docs/papers/VerifiableDataStructures.pdf) paper.
//
// The API supports adding new entries to the Merkle tree for a specific Log
// instance (identified by its log_id) in two modes:
// - For a normal log, new leaf entries are queued up for subsequent
// inclusion in the log, and the leaves are assigned consecutive leaf_index
// values as part of that integration process.
// - For a 'pre-ordered log', new entries have an already-defined leaf
// ordering, and leaves are only integrated into the Merkle tree when a
// contiguous range of leaves is available.
//
// The API also supports read operations to retrieve leaf contents, and to
// provide cryptographic proofs of leaf inclusion and of the append-only nature
// of the Log.
//
// Each API request also includes a charge_to field, which allows API users
// to provide quota identifiers that should be "charged" for each API request
// (and potentially rejected with codes.ResourceExhausted).
//
// Various operations on the API also allows for 'server skew', which can occur
// when different API requests happen to be handled by different server instances
// that may not all be up to date. An API request that is relative to a specific
// tree size may reach a server instance that is not yet aware of this tree size;
// in this case the server will typically return an OK response that contains:
// - a signed log root that indicates the tree size that it is aware of
// - an empty response otherwise.
service TrillianLog {
// QueueLeaf adds a single leaf to the queue of pending leaves for a normal
// log.
rpc QueueLeaf(QueueLeafRequest) returns (QueueLeafResponse) {}
// GetInclusionProof returns an inclusion proof for a leaf with a given index
// in a particular tree.
//
// If the requested tree_size is larger than the server is aware of, the
// response will include the latest known log root and an empty proof.
rpc GetInclusionProof(GetInclusionProofRequest)
returns (GetInclusionProofResponse) {}
// GetInclusionProofByHash returns an inclusion proof for any leaves that have
// the given Merkle hash in a particular tree.
//
// If any of the leaves that match the given Merkle has have a leaf index that
// is beyond the requested tree size, the corresponding proof entry will be empty.
rpc GetInclusionProofByHash(GetInclusionProofByHashRequest)
returns (GetInclusionProofByHashResponse) {}
// GetConsistencyProof returns a consistency proof between different sizes of
// a particular tree.
//
// If the requested tree size is larger than the server is aware of,
// the response will include the latest known log root and an empty proof.
rpc GetConsistencyProof(GetConsistencyProofRequest)
returns (GetConsistencyProofResponse) {}
// GetLatestSignedLogRoot returns the latest log root for a given tree,
// and optionally also includes a consistency proof from an earlier tree size
// to the new size of the tree.
//
// If the earlier tree size is larger than the server is aware of,
// an InvalidArgument error is returned.
rpc GetLatestSignedLogRoot(GetLatestSignedLogRootRequest)
returns (GetLatestSignedLogRootResponse) {}
// GetEntryAndProof returns a log leaf and the corresponding inclusion proof
// to a specified tree size, for a given leaf index in a particular tree.
//
// If the requested tree size is unavailable but the leaf is
// in scope for the current tree, the returned proof will be for the
// current tree size rather than the requested tree size.
rpc GetEntryAndProof(GetEntryAndProofRequest)
returns (GetEntryAndProofResponse) {}
// InitLog initializes a particular tree, creating the initial signed log
// root (which will be of size 0).
rpc InitLog(InitLogRequest) returns (InitLogResponse) {}
// AddSequencedLeaves adds a batch of leaves with assigned sequence numbers
// to a pre-ordered log. The indices of the provided leaves must be contiguous.
rpc AddSequencedLeaves(AddSequencedLeavesRequest)
returns (AddSequencedLeavesResponse) {}
// GetLeavesByRange returns a batch of leaves whose leaf indices are in a
// sequential range.
rpc GetLeavesByRange(GetLeavesByRangeRequest)
returns (GetLeavesByRangeResponse) {}
}
// ChargeTo describes the user(s) associated with the request whose quota should
// be checked and charged.
message ChargeTo {
// user is a list of personality-defined strings.
// Trillian will treat them as /User/%{user}/... keys when checking and
// charging quota.
// If one or more of the specified users has insufficient quota, the
// request will be denied.
//
// As an example, a Certificate Transparency frontend might set the following
// user strings when sending a QueueLeaf request to the Trillian log:
// - The requesting IP address.
// This would limit the number of requests per IP.
// - The "intermediate-<hash>" for each of the intermediate certificates in
// the submitted chain.
// This would have the effect of limiting the rate of submissions under
// a given intermediate/root.
repeated string user = 1;
}
message QueueLeafRequest {
int64 log_id = 1;
LogLeaf leaf = 2;
ChargeTo charge_to = 3;
}
message QueueLeafResponse {
// queued_leaf describes the leaf which is or will be incorporated into the
// Log. If the submitted leaf was already present in the Log (as indicated by
// its leaf identity hash), then the returned leaf will be the pre-existing
// leaf entry rather than the submitted leaf.
QueuedLogLeaf queued_leaf = 2;
}
message GetInclusionProofRequest {
int64 log_id = 1;
int64 leaf_index = 2;
int64 tree_size = 3;
ChargeTo charge_to = 4;
}
message GetInclusionProofResponse {
// The proof field may be empty if the requested tree_size was larger
// than that available at the server (e.g. because there is skew between
// server instances, and an earlier client request was processed by a
// more up-to-date instance). In this case, the signed_log_root
// field will indicate the tree size that the server is aware of, and
// the proof field will be empty.
Proof proof = 2;
SignedLogRoot signed_log_root = 3;
}
message GetInclusionProofByHashRequest {
int64 log_id = 1;
// The leaf hash field provides the Merkle tree hash of the leaf entry
// to be retrieved.
bytes leaf_hash = 2;
int64 tree_size = 3;
bool order_by_sequence = 4;
ChargeTo charge_to = 5;
}
message GetInclusionProofByHashResponse {
// Logs can potentially contain leaves with duplicate hashes so it's possible
// for this to return multiple proofs. If the leaf index for a particular
// instance of the requested Merkle leaf hash is beyond the requested tree
// size, the corresponding proof entry will be missing.
repeated Proof proof = 2;
SignedLogRoot signed_log_root = 3;
}
message GetConsistencyProofRequest {
int64 log_id = 1;
int64 first_tree_size = 2;
int64 second_tree_size = 3;
ChargeTo charge_to = 4;
}
message GetConsistencyProofResponse {
// The proof field may be empty if the requested tree_size was larger
// than that available at the server (e.g. because there is skew between
// server instances, and an earlier client request was processed by a
// more up-to-date instance). In this case, the signed_log_root
// field will indicate the tree size that the server is aware of, and
// the proof field will be empty.
Proof proof = 2;
SignedLogRoot signed_log_root = 3;
}
message GetLatestSignedLogRootRequest {
int64 log_id = 1;
ChargeTo charge_to = 2;
// If first_tree_size is non-zero, the response will include a consistency
// proof between first_tree_size and the new tree size (if not smaller).
int64 first_tree_size = 3;
}
message GetLatestSignedLogRootResponse {
SignedLogRoot signed_log_root = 2;
// proof is filled in with a consistency proof if first_tree_size in
// GetLatestSignedLogRootRequest is non-zero (and within the tree size
// available at the server).
Proof proof = 3;
}
message GetEntryAndProofRequest {
int64 log_id = 1;
int64 leaf_index = 2;
int64 tree_size = 3;
ChargeTo charge_to = 4;
}
message GetEntryAndProofResponse {
Proof proof = 2;
LogLeaf leaf = 3;
SignedLogRoot signed_log_root = 4;
}
message InitLogRequest {
int64 log_id = 1;
ChargeTo charge_to = 2;
}
message InitLogResponse {
SignedLogRoot created = 1;
}
message AddSequencedLeavesRequest {
int64 log_id = 1;
repeated LogLeaf leaves = 2;
ChargeTo charge_to = 4;
}
message AddSequencedLeavesResponse {
// Same number and order as in the corresponding request.
repeated QueuedLogLeaf results = 2;
}
message GetLeavesByRangeRequest {
int64 log_id = 1;
int64 start_index = 2;
int64 count = 3;
ChargeTo charge_to = 4;
}
message GetLeavesByRangeResponse {
// Returned log leaves starting from the `start_index` of the request, in
// order. There may be fewer than `request.count` leaves returned, if the
// requested range extended beyond the size of the tree or if the server opted
// to return fewer leaves than requested.
repeated LogLeaf leaves = 1;
SignedLogRoot signed_log_root = 2;
}
// QueuedLogLeaf provides the result of submitting an entry to the log.
// TODO(pavelkalinnikov): Consider renaming it to AddLogLeafResult or the like.
message QueuedLogLeaf {
// The leaf as it was stored by Trillian. Empty unless `status.code` is:
// - `google.rpc.OK`: the `leaf` data is the same as in the request.
// - `google.rpc.ALREADY_EXISTS` or 'google.rpc.FAILED_PRECONDITION`: the
// `leaf` is the conflicting one already in the log.
LogLeaf leaf = 1;
// The status of adding the leaf.
// - `google.rpc.OK`: successfully added.
// - `google.rpc.ALREADY_EXISTS`: the leaf is a duplicate of an already
// existing one. Either `leaf_identity_hash` is the same in the `LOG`
// mode, or `leaf_index` in the `PREORDERED_LOG`.
// - `google.rpc.FAILED_PRECONDITION`: A conflicting entry is already
// present in the log, e.g., same `leaf_index` but different `leaf_data`.
google.rpc.Status status = 2;
}
// LogLeaf describes a leaf in the Log's Merkle tree, corresponding to a single log entry.
// Each leaf has a unique leaf index in the scope of this tree. Clients submitting new
// leaf entries should only set the following fields:
// - leaf_value
// - extra_data (optionally)
// - leaf_identity_hash (optionally)
// - leaf_index (iff the log is a PREORDERED_LOG)
message LogLeaf {
// merkle_leaf_hash holds the Merkle leaf hash over leaf_value. This is
// calculated by the Trillian server when leaves are added to the tree, using
// the defined hashing algorithm and strategy for the tree; as such, the client
// does not need to set it on leaf submissions.
bytes merkle_leaf_hash = 1;
// leaf_value holds the data that forms the value of the Merkle tree leaf.
// The client should set this field on all leaf submissions, and is
// responsible for ensuring its validity (the Trillian server treats it as an
// opaque blob).
bytes leaf_value = 2;
// extra_data holds additional data associated with the Merkle tree leaf.
// The client may set this data on leaf submissions, and the Trillian server
// will return it on subsequent read operations. However, the contents of
// this field are not covered by and do not affect the Merkle tree hash
// calculations.
bytes extra_data = 3;
// leaf_index indicates the index of this leaf in the Merkle tree.
// This field is returned on all read operations, but should only be
// set for leaf submissions in PREORDERED_LOG mode (for a normal log
// the leaf index is assigned by Trillian when the submitted leaf is
// integrated into the Merkle tree).
int64 leaf_index = 4;
// leaf_identity_hash provides a hash value that indicates the client's
// concept of which leaf entries should be considered identical.
//
// This mechanism allows the client personality to indicate that two leaves
// should be considered "duplicates" even though their `leaf_value`s differ.
//
// If this is not set on leaf submissions, the Trillian server will take its
// value to be the same as merkle_leaf_hash (and thus only leaves with
// identical leaf_value contents will be considered identical).
//
// For example, in Certificate Transparency each certificate submission is
// associated with a submission timestamp, but subsequent submissions of the
// same certificate should be considered identical. This is achieved
// by setting the leaf identity hash to a hash over (just) the certificate,
// whereas the Merkle leaf hash encompasses both the certificate and its
// submission time -- allowing duplicate certificates to be detected.
//
//
// Continuing the CT example, for a CT mirror personality (which must allow
// dupes since the source log could contain them), the part of the
// personality which fetches and submits the entries might set
// `leaf_identity_hash` to `H(leaf_index||cert)`.
//
// TODO(pavelkalinnikov): Consider instead using `H(cert)` and allowing
// identity hash dupes in `PREORDERED_LOG` mode, for it can later be
// upgraded to `LOG` which will need to correctly detect duplicates with
// older entries when new ones get queued.
bytes leaf_identity_hash = 5;
// queue_timestamp holds the time at which this leaf was queued for
// inclusion in the Log, or zero if the entry was submitted without
// queuing. Clients should not set this field on submissions.
google.protobuf.Timestamp queue_timestamp = 6;
// integrate_timestamp holds the time at which this leaf was integrated into
// the tree. Clients should not set this field on submissions.
google.protobuf.Timestamp integrate_timestamp = 7;
}