Meta Data Checksums in Shared Records

Each metadata entry in Shared Records is stored in a file name with the following format: <Title>_<PaddedSequenceNumber>_<RollingChecksum>.

The rolling checksum is used to provide additional security and verifiability of the entries. Since Shared Records metadata is an append only scheme, no metadata entry should ever be modified. Adding this checksum ensures that if a metadata entry is changed the shared records server will know about it and no longer serve the metadata, if appropriate.

Computing the Checksum

The checksum is calculated as follows:

Checksum = SHA1Hash(ASCIIToBytes(<Previous Checksum> + <Metadata Header String>))

Where:

Checksum

The checksum we are computing for this metadata entry.

SHA1Hash

A function that calculates the SHA-1 hashcode of a set of bytes.

ASCIIToBytes

A function that converts a string to bytes using ASCII encoding.

Previous Checksum

The previous checksum retrieved from the last metadata entry based on sequence number, or an empty string if this is the first entry.

Metadata Header String

A string retrieved from the metadata entry that contains the headers below in order. Each header should be of the form "<Header>: <Value>" and headers should be separated by newlines. The string should also end with a single newline.

Headers

Header	Value	Required?
x-amz-meta-record-uid	The Record’s ID	Yes
x-amz-meta-title	The entry’s title	Yes
x-amz-meta-user-id	The creator of the entry	No
Content-Type	The content type of the entry	No
x-amz-meta-prev-entry	The previous metadata entry	If not the first entry
x-amz-meta-signature	A signature on the metadata	No
ETag	The MD5 hash of the entry’s body	Yes

Using the Shared Records code to Calculate Checksums

The Shared Records libraries provide convenient methods for calculating rolling checksums so that developers don't have to worry about the details of how the checksums are calculated.

Java API

The following methods are useful for calculating checksums using the Java library.

MetaDataEntry Methods

	/**
	 * Gets the headers for this object to be used in calculating the Checksum 
	 * 
	 * @return the checksum string
	 */

	public String getChecksumHeaderString()

MetaDataUtils Methods

	/**
	 * Calculate the checksum of the metadata entry based on the previous checksum 
	 * and the data inside the entry.
	 * @param previousRollingChecksum the previous checksum, or an empty string if
	 * there is no previous checksum
	 * @param mde The entry to compute the checksum for
	 * @return the new checksum.
	 */
	public static String computeRollingChecksum(String previousRollingChecksum, MetaDataEntry mde)

C# API

The following methods are used by the C# platform to compute checksums.

MetaDataEntry Methods

        /// <summary>
        /// Returns a string containing the headers used to calculate the rolling checksum.
        /// The resultant string is of the form:
        /// header-key1: value1
        /// header-key2: value2
        /// etc...
        /// </summary>
        /// <returns>A string containing all the non-empty, non-null headers used to compute the checksum.</returns>
        public string GetChecksumHeadersString()

        /// <summary>
        /// Computes the rolling checksum for this entry, based on the previous checksum
        /// and the current state of the MetaData headers.
        /// </summary>
        /// <param name="previousChecksum">The checksum of the last MetaDataEntry to be added, or the empty string if
        /// this is the first entry to be added to its parent record.</param>
        /// <returns>The hex string representation of the checksum.</returns>
        public string ComputeRollingChecksum(string previousChecksum)