- Sigsum-general - Sigsum mailing lists

Re: Witnessing for high-traffic log
by Hayden Blauzvern 06 Feb '24

06 Feb '24

Thanks Rasmus! I've cc'd the list and added Bob who's interested in this topic too. What submit latency are you willing to accept? I'm asking because > depending on if you need ~1s or ~10s will influence the options. > I'd like to keep this latency as low as possible. It would be a breaking change across the ecosystem if we upped latency to ~10s, as I'm assuming clients have not configured their timeouts to expect this high of a latency. That's not to say we couldn't make this change, as we could provide a different API, I'd just like to explore a low latency initially. I.e., the log can keep track of a witness' latest state X, then provide > to the witness a new checkpoint Y and a consistency proof that is valid > from X -> Y. If all goes well, the witness returns its cosignature. If > they are out of sync, the log needs to try again with the right state. Assuming that all witnesses are responsive and maintain the same state, this could work. Keeping track of N different witnesses is doable, but I think it's likely they would get out of sync, e.g. a request to cosign a checkpoint times out but the witness still verifies and persists the checkpoint. This isn't a blocker though, it's just an extra call if needed. The current plan for Sigsum is to accept up to T seconds of logging > latency, where T is in the order of 5-10s. Every T seconds the log > selects the current checkpoint, then it collects as many cosignatures as > possible before making the result available and starting all over again. This seems like the most sensible approach assuming that latency can be accepted by the ecosystem. Batching entries is something we've discussed before, there's other performance benefits besides witnessing. > An alternative implementation of the same witness protocol would be as > follows: always be in the process of creating the next witnessed > checkpoint. I.e., as soon as one finalized a witnessed checkpoint, > start all over again because the log's tree already moved forward. To > keep the latency down, only collect the minimum number of cosignatures needed to satisfy all trust policies that the log's users depend on. This makes sense, though I think adding some latency as suggested above makes this more straightforward. One detail, which may not be relevant depending on your order of operations, is that we just need to confirm that the inclusion proof returned will be based on the cosigned checkpoint. Currently our workflow is first requesting an inclusion proof for the latest tree head, then signing the tree head. On Fri, Feb 2, 2024 at 3:37 AM Rasmus Dahlberg <rgdd(a)glasklarteknik.se> wrote: > Hi Hayden, > > Exciting that you're exploring this are, answers inline! > > On Thu, Feb 01, 2024 at 01:05:48PM -0800, Hayden Blauzvern wrote: > > Hey y'all! I was reading up on Sigsum docs and witnessing and had a > > question about if or how you're handling logs with significant traffic. > > > > Context is I've been looking at improving our witnessing story with > > Sigstore and exploring the viability of the bastion-based witnessing > > approach. Currently, the Sigstore log does no batching of entry uploads, > > and so the tree head/checkpoint is frequently updated. Consequently this > > means that two witnesses are very unlikely to witness the same > checkpoint. > > To solve this, we added a 'stable' checkpoint, one that is published > every > > X minutes (5 currently). Witnesses are expected to compute consistency > > proofs off that checkpoint so that multiple witnesses verify the same > > checkpoint. > > Sounds similar the initial witness protocol we used: the log makes > available a checkpoint for some time, and witnesses poll to cosign it. > > We moved away from this communication pattern to solve two problems: > > 1. High submit latency, which is the issue you're experiencing. > 2. Ensure logs without publicly reachable endpoints are not excluded. > > While reworking this, we also tried to keep as many of the properties we > liked with the old protocol. For example, the bastion host stems from > the nice property that witnesses can be pretty locked down behind a NAT. > > > > > I've been exploring the bastion-based approach where for each entry or > tree > > head update, the log requests cosignatures from a set of witnesses. What > > I'm pondering now is how to deal with a log that frequently updates its > > tree head due to frequent new entries. > > One solution is to batch entries for a long enough period, let's say 1 > > minute, so that the log can fetch cosignatures from a quorum of witnesses > > while accounting for some latency. But this is not our preferred user > > experience, to have signers wait that long. > > Lowering the batch to 1 second would solve the UX issue. > > What submit latency are you willing to accept? I'm asking because > depending on if you need ~1s or ~10s will influence the options. > > > However now > > there's an issue for updating a witness's checkpoint. Using the API > Filippo > > has documented for the witness, the log makes two requests to the > witness: > > One for the latest witness checkpoint, one to provide the log's new > > checkpoint. > > The current witness protocol allows the log to collect a cosignature > from a witness in a single API call, see the add-tree-head endpoint: > > > https://git.glasklar.is/sigsum/project/documentation/-/blob/d8de0eeebbb5bb0… > > (Warning: the above API document is being reworked and moved to C2SP. > The new revision will revolve around checkpoint names and encodings. > You'll find links to all the decided proposals on www.sigsum.org/docs.) > > I.e., the log can keep track of a witness' latest state X, then provide > to the witness a new checkpoint Y and a consistency proof that is valid > from X -> Y. If all goes well, the witness returns its cosignature. If > they are out of sync, the log needs to try again with the right state. > > > This seemingly would not work with a high-volume log since the > > witness's latest checkpoint would update too frequently. > > > > Did you have any thoughts on how to handle this? > > The current plan for Sigsum is to accept up to T seconds of logging > latency, where T is in the order of 5-10s. Every T seconds the log > selects the current checkpoint, then it collects as many cosignatures as > possible before making the result available and starting all over again. > > The rationale is: a witness that is online will be able to respond in > 5-10s, so waiting longer than that will not really do much. I.e., the > witness is either online and responding or it isn't. So: under normal > circumstances one would expect cosignatures from all reliable witnesses. > > An alternative implementation of the same witness protocol would be as > follows: always be in the process of creating the next witnessed > checkpoint. I.e., as soon as one finalized a witnessed checkpoint, > start all over again because the log's tree already moved forward. To > keep the latency down, only collect the minimum number of cosignatures > needed to satisfy all trust policies that the log's users depend on. > > For example, if you're opinionated and say users should rely on 10 > selected witnesses with a 3-of-10 policy; the log server can publish the > next checkpoint as soon as it received cosignatures from 3 witnesses. > > Both approaches work, but depending on which one you choose the > properties and complexity will be slightly different. Avoiding to hash > out that analysis here in order to keep this initial answer brief, but > if you need the ~1s latency the second option should get you close. > > By the way, would it be OK to @CC the sigsum-general list? Pretty sure > this is a conversation other folks would be interested in as well! > > -Rasmus >

3 2

Any value in storing tree head signatures locally?
by Niels Möller 21 Nov '23

21 Nov '23

Hi, I'm looking at my draft MR for persisting monitor state (https://git.glasklar.is/sigsum/core/sigsum-go/-/merge_requests/152). There's a bit of extra complexity there because the monitor stores treeheads *including* signature. Those signatures are verified when it is restarted and state is read back from disk. It's not clear to me that provides any real value, though. Do we lose anything if we verify signatures when tree heads are received from the network, and then discard the signatures and just trust local system integrity? One attack I'm thinking of: Say leaf 17 records some malicious act that the monitor would flag. If the monitor can be taken down when it has processed the log up to leaf 15, and then have it's state fast-forwarded so that when it is restarted, it thinks it already have looked at everything up to leaf 20, that's a problem. However, (i) verifying the tree head signature doesn't help; since attacker can get a properly signed later tree head by just asking the log, and (ii) if the attacker is control of the local storage, monitoring is broken anyway. Almost the same question applies to witnesses, which also needs to persist latest observed and cosigned tree head. My witness implementation stores the log's signature and its own cosignature, and verifies both when state is loaded at restart. The relevant attack here is forcing the state backwards, rather than forwards, but the signatures appear to not provide any protection against that attack either. And in either scenarios, in case the treehead is modified in any other way than to a different valid treehead in the past or in the future, that should result in a consistency error when the monitor/witness received the next treehead from the log. Am I missing something, or can we just drop local storage of tree head signatures? Regards, /Niels

2 3

On k-of-n witnessing
by Niels Möller 17 Nov '23

17 Nov '23

Not sure if this is already well known and documented somewhere, but I'd like to sort out the implication of dishonest witnesses. Say there are a total of n witnesses, m of which are honest. The other n-m are willing to participate in either denial of service (refuse to cosign anything) or split-view attacks (cosign multiple inconsistent tree heads). Assume client policy requires at least k valid cosignatures. To thwart denial of service attacks, we must have k <= m. For a split-view attack, the attacker wants two clients (with the same policy) to accept two distincts views of the log. It seems the best the attacker can do is to have each view the tree cosigned by half of the honest witnesses and all of the dishonest witnesses, so that the number of valid cosignatures on each view is n - floor(m/2), n - ceil(m/2) The attack succeeds if both are accepted, and it is thwarted if k > n - ceil(m/2) We would like to have at least one value of k that is secure, and that would have to be k = m, and then we need k > n - ceil(k/2) which should be equivalent (if I get the details of ceil and floor logic right) to k > 2n/3 and k >= 1 + floor(2*n/3) So a little table, n 1 2 3 4 5 6 7 8 9 10 min k 1 2 3 3 4 5 5 6 7 7 The important point here, is that if one uses a k-of-n policy with smaller k than in this table, say, 5-of-8 witnesses, then it is *not* sufficient with 5 honest witnesses to reject a split view, since we could have a split view with 5 and 6 cosignatures on each view. And 6 honest witnesses isn't enough either, we need at least 7, i.e., we can tolerate at most a single dishonest witness. It also seems worth noting that we need at least 4 witness before we can have any redundancy. Regards, /Niels

1 0

ANNOUNCE: Sigsum Log Server Protocol version v1.0.0
by Niels Möller 08 Nov '23

08 Nov '23

The Sigsum team is happy to make the first official release, version v1.0.0, of the Sigsum Log Server Protocol. This is the protocol used to submit new entries to a Sigsum log, and to query the log for cosigned tree heads, stored entries, and for inclusion and consistency proofs. See https://www.sigsum.org/ for more information about the Sigsum project. NEWS file entry for this release is appended below. This release of the specification can be read at https://git.glasklar.is/sigsum/project/documentation/-/blob/log.md-release-…. The specification will also be published on https://www.sigsum.org/. /The Sigsum team NEWS for log.md, version v1.0.0 This is the first official release of the Sigsum Log Server Protocol, i.e., the file log.md. This release is identified by the git tag log.md-release-v1.0.0. It is intended to be stable, and there are no planned breaking changes. See README.md for information on development and release processes for Sigsum specifications. The protocol is implemented by the log-go server v0.14.1 announced recently, see https://git.glasklar.is/sigsum/core/log-go/-/blob/v0.14.1/NEWS, and by the command line tools at https://git.glasklar.is/sigsum/core/sigsum-go, current version being v0.6.1.

2 1

ANNOUNCE: log-go v0.14.1
by Niels Möller 01 Nov '23

01 Nov '23

The Sigsum team is happy to release a new version of the log-go log server software, version tag v0.14.1, succeeding the previous release v0.9.0. The source code for the release can be checked out from the git repository as git clone -b v0.14.1 https://git.glasklar.is/sigsum/core/log-go.git or installed using go install sigsum.org/log-go/cmd/sigsum-log-primary@v0.14.1 This release updates the log to the version 1 Sigsum protocol. It also adds back support for witness cosigning, following an interim witness protocol. See NEWS file for details on changes and how to upgrade, excerpt below. If you find any bugs, please report them here on the sigsum-general mailing list or open an issue on GitLab in the log-go repository: https://git.glasklar.is/sigsum/core/log-go/ The expectations and intended use of the log server software is documented in the log-go RELEASES file. You will also find more information about the overall release process there, see https://git.glasklar.is/sigsum/core/log-go/-/blob/main/RELEASES.md. / The Sigsum team NEWS for log-go v0.14.1 This release updates the log server to implement the v1 sigsum protocols [1]. The log protocol, i.e., the protocol for querying a log and submitting new entries, is now considered stable: there are no plans for incompatible changes, and future updates will consider transitions carefully. However, the witness protocol, used by logs to interact with witnesses, is under development, and will likely be replaced in later versions. Upgrading existing log services from the previous release, v0.9.0, is automatic, see below for the implications of upgrading. However, downgrading is *NOT* tested nor supported. Downgrading would require manual replacement of the signed-tree-head file by restoring it from a backup or recreating it using sigsum-mktree. This release has been tested to work together with: * Command line tools (most importantly sigsum-submit and sigsum-verify) https://git.glasklar.is/sigsum/core/sigsum-go, tag v0.6.1. * The prototype witness, https://git.glasklar.is/sigsum/core/sigsum-py, tag v0.1.1. * The litetlog witness, https://github.com/FiloSottile/litetlog, tag v0.1.1. New features: * Implemented a new interim witness protocol [2], where a log queries witnesses for cosignatures. Witnesses to use are configured using a sigsum policy file. Further changes to the log <-> witness protocol are planned. Incompatible changes: * Tree head serialization used for signatures and cosignatures changed to use a checkpoint-compatible [3] text serialization. * Leaf signatures as well as submit token signatures changed to no longer use SSH signature format. Notes on upgrading: * Existing logs can be upgraded. When loading a log server's signed-tree-head file, the server accepts either a sigsum v0 tree head signature, as created by log-go v0.9.0, or a v1 signature, as created by the current version. * An upgraded log will publish tree heads signed according to the v1 protocol, and accept new leaves signed according to the v1 protocol. * Old leaves submitted according to the v0 protocol obviously stay in the tree, but they will appear invalid to anyone who has the submitter's public key and attempts to verify the leaf signature according to the v1 specification. * Old sigsum proofs can still be verified using old tools (since verification is purely offline, the log server is not involved at all). However, attempting to create a new proof for an old leaf will fail. Miscellaneous: * An intermediate version, tagged v0.13.0, included an explicit "v1" signature version on the cosignature lines. This protocol change was reverted for v0.14.x. Version v0.13.0 was never announced or properly released, but in case anyone is nevertheless running v0.13.0, upgrading to v0.14.1 is expected to work. The only user-visible change is the removal of cosignature version, which means that any client software and witnesses written to interop with log servers running v0.13.0 will need upgrading as well. [1] https://git.glasklar.is/sigsum/project/documentation/-/blob/main/log.md [2] https://git.glasklar.is/sigsum/project/documentation/-/blob/4ee138f1294f9c1… [3] https://github.com/transparency-dev/formats/blob/main/log/README.md#checkpo…

2 2

Sketch of tkey apps for signing keys with backup
by Niels Möller 21 Sep '23

21 Sep '23

Tkey for signing keys. Motivation: Sigsum needs long-lived signing keys, with backup in case the device holding the signing key (and maybe the rest of the site as well) is destroyed. Same may apply to other long lived keys, e.g., close to the top of a cert hierarchy/delegation chain. 1. Signer app: Recieve an encrypted signing key, encrypted to the key's unique key using some public key crypto algorithm. Decrypts the signing key, and acts as signing oracle. Also needs a way to advertise its own pubkey, as well as the pubkey corresponding to the signing key. 2. Keygen app: Takes a list of public keys. Generates a new signing key, and encrypts it to each of the public keys. 3. Export app: Takes public keys and an encrypted signing key. Decrypts using its own unique key, and reencrypts to each of the public keys. Also needs a way to advertise its own pubkey. (Similar role as a yubihsm with key export enabled, but with "keywrap" done using public keys rather than a symmetric key). Usage scenario: Get at least three tkeys, two intended as backups, the other(s) as signing oracles. Extract the public keys for each key and the respective intended app (signer or exporter). Load the key-gen app on any of the keys, and give it all those pubkeys. Store the resulting encrypted blob reliably. Lock up the two backup/export keys. Attach a signing key to the machine intended to do signing, and provide the encrypted blob and the signing app on that machine. When any one of the keys breaks, take one of the export keys out of the safe, and use it to reencrypt the signing key to a replacement key of the appropriate type. Extension 1 (technically quite easy, I think): Add a symmetric key/passphrase, required for operation of the keys, either a single one, or separate for each key. Would be particularly relevant for access to the export keys, and it would also make sense to apply k-of-n secret sharing of the secret needed to operate the export keys. Extension 2 (hand waving): Have the signer app require a proof of logging of the act of exporting the signing key to that particular key. /Niels

1 0

Private key backup
by Niels Möller 04 Apr '23

04 Apr '23

Hi, I wrote up some notes and ideas on how to do backups of private keys, which is needed for sigsum primary-secondary failover. I would hope the way I propose (if it actually works) could be implemented on a Tillitis key. I have no idea if something like that is supported by yubico hsm, but I think Linus is investigating. See https://git.glasklar.is/sigsum/project/documentation/-/blob/main/archive/20…, all comments and feedback appreciated. /Niels

2 1

First release of the Sigsum log software and ansible roles
by Morten Linderud 31 Mar '23

31 Mar '23

We are happy to announce the first release of the sigsum log server software and Ansible role collection! This means that the project is ready for wider deployment, and that we are committed to provide well-documented upgrade paths when releasing future versions. For background, recall that Sigsum makes signed checksums transparent. This makes it possible to detect malicious and unintended key-usage. The ansible role collection aims to make it easy for organizations to host sigsum logs on their own infrastructure, including setup of mariadb and data replication between a log instance's primary and secondary nodes. The v1.0.0 Ansible release makes use of the following versions: - log-go v0.9.0 - sigsum-go v0.1.23 - Trillian v1.5.1 For more information relating to the log server architecture and how to get started with our ansible collection, see: https://git.glasklar.is/sigsum/admin/ansible https://git.glasklar.is/sigsum/core/log-go/-/blob/main/doc/readme.md We recommend to also take a look at the Sigsum roadmap: https://git.glasklar.is/sigsum/project/documentation/-/blob/main/archive/20… https://git.glasklar.is/groups/sigsum/-/milestones Of note is that the sigsum protocol is still at version 0. While we are not expecting any major changes at this point, an upcoming release will extend the log software with a witness cosigning protocol that is still in-progress. Change logs and the information necessary to perform upgrades will be provided from here on. However, it is not recommended for end-users to fail-close on sigsum logging before protocol version 1. Please report issues and request support on our GitLab issue tracker. We are also available on irc.oftc.net and Matrix in room #sigsum. More informasion can also be found on our webpage: https://www.sigsum.org/ Cheers, The Sigsum team

1 0

Meaning of a single cosignature?
by Niels Möller 17 Mar '23

17 Mar '23

Hi, this is a question that I hope is convered in the literature on transparency logs, but what meaning do we assign to a single, isolated, cosignature? If we have a *sequence* of cosigned tree heads (signed by a particular witness, ordered by increasing tree size), then each cosignature says that all tree leafs in previous cosigned trees are present in later trees. But a sigsum client only sees a single cosigned tree head, not the sequence. In particular, consider the case of the first cosignature of a particular witness, with no history. So what exactly does that signature mean? I suspect there are maybe some underlying assumptions or policys that should be spelled out somewhere. Is this an reasonably accurate description? When I see a leaf node accompanied by a cosigned treehead and an inclusion proof leading up to that head, then I expect that the witness will raise some alarm if, sometime in the future, the log attempts to publish a tree head where the leaf no longer is present? In this case, a single cosignature doesn't make any claim about the state of the log, it just states that the witness has observed this state, and hence that the witness has the information needed to detect future inconsistencies. More concisely: An isolated cosignature (without history) does not make a claim about the state of the log, it makes a claim about the state of the witness. Does that make sense? Regards, /Niels

2 4

Re: We include the checksum in tree_leaf. But what is it used for?
by Niels Möller 17 Nov '22

17 Nov '22

Rasmus Dahlberg <rasmus.dahlberg(a)glasklarteknik.se> writes: >> I see one problem with this, though. The monitor can't simply use the >> ssh-keygen command to verify the signature, since that command expects >> to get the *message* as input, not the hash thereof. Which kind-of >> defeats the idea of piggybacking on ssh tools. > > I disagree. The value of piggy-backing on SSH tooling is for the signer > who can access their private key with good solutions that already exist. I don't have a very strong opinion, maybe it's "only" a matter of documentation. If we say "sigsum uses the ssh signature format", I would expect that to mean that we use it as a black box with inputs and output according to the spec, but that's not quite what we do. We could say that we have checksum = H(message) and that the essence of the signature is to sign checksum, so that signature can be verified given only the public key and checksum (i.e., without knowing message). And that we do signatures in a way that produces an identical signature as if feeding message as the input of ssh-keygen -Y sign --hashalg=sha256, or M = message in its spec. >> message=SHA3(data) ; application layer > > Note that message must be exactly 32 bytes in Sigsum. So, you wouldn't > be able to use SHA3 here Side note: SHA3_256 is part of the SHA3 standard. > I see your point if it is a desired property to verify leaf signatures > in isolation with ssh-keygen. Would you say that the complexity is > decreased, about the same, or increased if this change was proposed? I think conceptual complexity would be decreased slightly, by which I mean roughly that it will be slightly easier to document and explain. For implementation complexity, not much difference (assuming that the only value we see in actually using the ssh-keygen tool for handling signatures is for the private key operations). Regards, /Niels

2 2

2024

2023

2022

2021

Sigsum-general