rmp_serde : Timestamp Question

[errantmind]

Hello, I am porting some messagepack code from another language and need to deserialize Msgpack's Timestamp type. Are there any resources on how to do this?

I'm guessing I need to write some kind of custom deserializer for this value through Serde but wanted to make sure there wasn't an easier way first.

[kornelski]

A while ago @dani-garcia and @vmx used timestamps with rmp. Maybe they can help.

I think it would be nice if we could provide wrapper types or functions for serde(with= to encode timestamps to/from std SystemTime or Duration.

[errantmind]

Yea, if anyone can point to a resource, that would help me (and anyone else who searches for this in the future). If not, no worries, I can work around it.

[renato145]

Did anyone found an example o.o?

[renato145]

Ok, so in my case I wanted to get the amount of seconds and following https://github.com/msgpack/msgpack/blob/master/spec.md#timestamp-extension-type got this code working:

#[derive(Debug, Serialize, Deserialize)]
struct SomeStruct {
    #[serde(
        serialize_with = "serialize_mp_date_secs",
        deserialize_with = "deserialize_mp_date_secs"
    )]
    date: u64,
}

#[derive(Debug, Serialize, Deserialize)]
#[serde(rename = "_ExtStruct")]
pub struct ExtStruct((i8, ByteBuf));

fn u32_from_bytebuf(buf: &ByteBuf) -> Result<u32, TryFromSliceError> {
    let bytes = buf.as_slice().try_into()?;
    Ok(u32::from_be_bytes(bytes))
}

fn u64_from_bytebuf(buf: &ByteBuf) -> Result<u64, TryFromSliceError> {
    let bytes = buf.as_slice().try_into()?;
    Ok(u64::from_be_bytes(bytes))
}

pub fn serialize_mp_date_secs<S>(secs: &u64, serializer: S) -> Result<S::Ok, S::Error>
where
    S: Serializer,
{
    let data64 = *secs;
    let ext = if (secs >> 34) == 0 {
        if (data64 & 0xffffffff00000000) == 0 {
            // timestamp 32
            let data32 = data64 as u32;
            ExtStruct((-1, ByteBuf::from(data32.to_be_bytes())))
        } else {
            // timestamp 64
            ExtStruct((-1, ByteBuf::from(data64.to_be_bytes())))
        }
    } else {
        // timestamp 96
        let mut bytes = 0u32.to_be_bytes().to_vec();
        let mut secs = (data64 as i64).to_be_bytes().to_vec();
        bytes.append(&mut secs);
        ExtStruct((-1, ByteBuf::from(bytes)))
    };
    ext.serialize(serializer)
}

pub fn deserialize_mp_date_secs<'de, D>(deserializer: D) -> Result<u64, D::Error>
where
    D: Deserializer<'de>,
{
    let ExtStruct((ext_type, buf)) = Deserialize::deserialize(deserializer)?;
    if ext_type != -1 {
        return Err(de::Error::custom("Invalid extension type (!=-1)"));
    }

    let sec = match buf.len() {
        4 => {
            let sec = u32_from_bytebuf(&buf).map_err(|e| {
                de::Error::custom(format!("Failed to get u32 from bytebuf ({})", e))
            })?;
            sec as u64
        }
        8 => {
            let data64 = u64_from_bytebuf(&buf).map_err(|e| {
                de::Error::custom(format!("Failed to get u64 from bytebuf ({})", e))
            })?;
            data64 & 0x00000003ffffffff
        }
        12 => {
            u64_from_bytebuf(&buf)
                .map_err(|e| de::Error::custom(format!("Failed to get u64 from bytebuf ({})", e)))?
                + 4
        }
        n => {
            return Err(de::Error::custom(format!(
                "Invalid data len {n} (valid sizes are 4, 8 and 12)"
            )))
        }
    };

    Ok(sec)
}

[8192K]

Here's some code allowing for nanoseconds as well. To be used in the same way as above. The struct field has to have the Timestamp type.

use serde::{Serialize, Serializer, Deserialize, Deserializer};
use serde_bytes::ByteBuf;

#[derive(Debug, Serialize, Deserialize)]
#[serde(rename = "_ExtStruct")]
struct ExtStruct((i8, ByteBuf));

pub fn serialize_mp_timestamp<S>(timestamp: &Timestamp, serializer: S) -> Result<S::Ok, S::Error>
where
    S: Serializer,
{
    ExtStruct((-1, timestamp.to_bytes())).serialize(serializer)
}

pub fn deserialize_mp_timestamp<'de, D>(deserializer: D) -> Result<Timestamp, D::Error>
where
    D: Deserializer<'de>,
{
    let ExtStruct((ext_type, buf)) = Deserialize::deserialize(deserializer)?;
    if ext_type != -1 {
        return Err(serde::de::Error::custom("Invalid extension type (!=-1)"));
    }

    Timestamp::from_bytes(buf).map_err(|e| serde::de::Error::custom(e))
}

#[derive(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct Timestamp {
    pub seconds: u64,
    pub nanoseconds: u32,
}

impl Timestamp {
    fn from_bytes(bytes: ByteBuf) -> Result<Self, &'static str> {
        let bytes = bytes.into_vec();
        let len = bytes.len();
        let (seconds, nanoseconds) = match len {
            4 => (u32::from_be_bytes(bytes.try_into().unwrap()) as u64, 0),
            8 => {
                let data64 = u64::from_be_bytes(bytes.try_into().unwrap());
                (data64 & 0x00000003ffffffff, (data64 >> 34) as u32)
            }
            12 => {
                let (nanoseconds_bytes, seconds_bytes) = bytes.split_at(4);
                (
                    u64::from_be_bytes(seconds_bytes.try_into().unwrap()),
                    u32::from_be_bytes(nanoseconds_bytes.try_into().unwrap()),
                )
            }
            _ => return Err("Timestamp can only be created from 32, 64, or 96-bit byte objects"),
        };

        Ok(Self {
            seconds,
            nanoseconds,
        })
    }

    fn to_bytes(&self) -> ByteBuf {
        if (self.seconds >> 34) == 0 {
            let mut data = [0u8; 8];

            let data64 = (self.nanoseconds as u64) << 34 | self.seconds;

            if data64 & 0xFFFFFFFF00000000 == 0 {
                data[..4].copy_from_slice(&data64.to_be_bytes()[..4]);
                data[4..].fill(0);
            } else {
                data.copy_from_slice(&data64.to_be_bytes());
            }

            return ByteBuf::from(data);
        }

        let mut data = [0u8; 12];
        data[..4].copy_from_slice(&self.nanoseconds.to_be_bytes());
        data[4..].copy_from_slice(&self.seconds.to_be_bytes());

        ByteBuf::from(data)
    }
}

[junderw]

I am starting something in #351 to make dealing with Timestamps a bit easier.