Hi Hayden! Thank you for reaching out with this.
The requirements you describe seems analogous to what I (with my Go project hat) need for the Go Checksum Database, and was indeed a motivation for moving to a synchronous witness API.
My plan is what Rasmus described, except with somewhat more optimistic expectations of the latency of witnesses. 1. Incorporate a batch of new leaves, possibly holding the submission requests. 2. Sign a new checkpoint. 3. Send out parallel requests to all witnesses to cosign the checkpoint, over keep-alive connections. 4. As soon as enough cosignatures are returned, publish the checkpoint and release the client requests. 5. If a witness doesn't return by the time the next checkpoint is signed, ignore it for the next round(s). 6. If a witness times out, assume it kept the previous state. If that's incorrect, it will send a 409 Conflict response with its correct state (this is a recent API change based on Trust Fabric feedback) and can be contacted successfully in the next round. I believe all of that can happen in 1s. 500ms to batch leaves, 500ms for m-of-n witnesses to respond to a single HTTP request over an established connection. US west coast to EU RTT is <150ms, that leaves 350ms for the witness to do signature verify+sign and database read+write.