Skip to content

Iloggable

  • in net
  • 1 min read

An exercise in overcomplication

Ok, there really isn't a need for doing this, but since i'm already stuck on creating compact language independent binary representations, here's a quick struct with an int, a fixed sized string and a variable data field that implements a quick and dirty serialization to go along with it

/// <summary>
/// Example of completely manual serialization of a struct containing a  fixed
/// sized string and a variable sized data block
/// </summary>
struct FixedStringAndVariableData
{
    // the fixed size of the valueString
    const int VALUE_SIZE = 10;
    Int32 id;
    string valueString;
    byte[] data;

    public int Id
    {
        get { return id; }
        set { id = value; }
    }

    public int ValueSize
    {
        get { return VALUE_SIZE; }
    }

    public string Value
    {
        get
        {
            if( valueString == null )
            {
                valueString = "".PadRight(10,' ');
            }
            return valueString;
        }
        set
        {
            if( value == null )
            {
                valueString = "".PadRight(10,' ');
            }
            else if( value.Length < 10 )
            {
                valueString = value.PadRight(10,' ');
            }
            else if( value.Length > 10 )
            {
                valueString = value.Substring(0,10);
            }
        }
    }

    public byte[] Data
    {
        get { return data; }
        set { data = value; }
    }

    public FixedStringAndVariableData(byte[] pRaw)
    {
        id = BitConverter.ToInt32(pRaw,0);
        Int32 offset =  Marshal.SizeOf(id.GetType());
        valueString = Encoding.ASCII.GetString(pRaw,offset,VALUE_SIZE);
        offset += VALUE_SIZE;
        Int32 remainder = pRaw.Length - offset;
        data = new byte[remainder];
        for(int i=0;i<remainder;i++)
        {
            data[i] = pRaw[offset+i];
        }
    }

    public byte[] Serialize()
    {
        Int32 size = Marshal.SizeOf(id.GetType())+VALUE_SIZE+data.Length;
        IntPtr pnt = Marshal.AllocHGlobal(size);
        byte[] serialized = new byte[size];
        int position = 0;
        byte[] buffer = BitConverter.GetBytes(id);
        buffer.CopyTo(serialized,position);
        position += buffer.Length;
        buffer = Encoding.ASCII.GetBytes(this.Value);
        buffer.CopyTo(serialized,position);
        position += buffer.Length;
        data.CopyTo(serialized,position);
        return serialized;
    }
}

Ok, i'll get off this subject now :)

  • in net
  • 1 min read

Fixed arrays in structs

Figured out that the good old C

struct foo
{
  int many[10];
}

does not work in C#. Best description of workarounds I found are on Eric Gunnerson blog here. Ho hum. Of course, it's a problem of limited application, i.e. dealing with legacy binary data or sending binary packets across the net. Now i know.

  • in net
  • 1 min read

When casting is not enough

Dealing with raw bytes coming across the network has given me plenty of opportunity to figure out how to convert data from one type to another. As previously noted, the good old C method of memcpy'ing bytes into structs takes a bit more work in C#, but is reasonable enough for having the benefit of a GC.

However, when it comes to just taking 4 bytes and turning them into an Int32, i certainly didn't want to jump throuhg those hoops. And you don't have to. Between the static classes of Convert, BitConverter and Encoding.ASCII you can pretty much get any type into any other. This particular case it calls for:

int i = 123456789;
byte[] b = BitConverter.GetBytes(i);
int j = BitConverter.ToInt32(b,0);

I wonder if behind the scenes BitConverter does the whole pinning down of memory game or just does byte arithmetic.

Now, how to define fixed sized strings in structs... I mean i could just create a sub-struct that creates individual slots for each byte it can contain, but that seems as little cumbersome.

  • in net
  • 1 min read

DIY Serialization

Forgot about putting this up as promised. Of course, this is a very simplistic example with only one variable sized field. If you had more, you'd have to also serialize those field's sizes and reconstruction becomes a bit more complicated.

But the important part about this, is really the whole concept of pinning some memory down so that you can manipulate it directly and then releasing it back to the control of the Garbage collector. Pretty cool, really.

[StructLayout(LayoutKind.Sequential)]
struct DIYSerialize
{
  public Int32 Id;
  public byte[] Data;

  public DIYSerialize(Int32 id, string data)
  {
    this.Id = id;
    this.Data = Encoding.ASCII.GetBytes(data);
  }
  public DIYSerialize(byte[] Raw)
  {
    Int32 size = Raw.Length;
    IntPtr pnt = Marshal.AllocHGlobal(size);
    GCHandle pin = GCHandle.Alloc(pnt,GCHandleType.Pinned);
    Marshal.Copy(Raw,0,pnt,size);
    this = (DIYSerialize)Marshal.PtrToStructure(pnt,typeof(DIYSerialize));
    pin.Free();
  }
  public byte[] Serialize()
  {
    Int32 size = Marshal.SizeOf(Id.GetType()) + Data.Length;
    IntPtr pnt = Marshal.AllocHGlobal(size);
    GCHandle pin = GCHandle.Alloc(pnt);
    Marshal.StructureToPtr(this,pnt,false);
    byte[] d = new byte[size];
    Marshal.Copy(pnt,d,0,Data.Length);
    pin.Free();
    Marshal.FreeHGlobal(pnt);
    return d;
  }
}
  • in net, geek
  • 1 min read

Interop vs. Serialize

I was this close to just using ISerializable for by binary representation for networking. But then talking to n8, i decided that leaving the door open for java interop was important enough in Enterprise computing, that I couldn't ignore it.

So, I'm back to coming up with my packet structure in manner that i can easily serialize by hand. Right now the plan is something along the line of a fixed sized header followed by a variable sized gzip-Xml chunk and a variable sized raw data chunk. Both of the variable chunks can be 0 length. The header is used for expressing the purpose of the packet, and describing the offsets of the variable packets. The gzip-Xml is for data that the recipient is supposed to parse and act on, while the raw data packet will be just that, raw data that i sued to construct a complete file.

I need to dig up the test code i did for doing manual serialization in .NET. Really, just what would be done in C with a struct and memcpy, but of course, since in C# you don't have pointers and the memory is managed for you, it's a bit more of a hoop jumping exercise. The code is on another machine, so i'll leave that for another post.

  • in net
  • 1 min read

Fun with enum bitflags

I've previously learned about the enum [Flags()] attribute, but had forgotten usage and reference to it isn't the best, so i figured, iput this down before it slips my mind again.

Say you want a compact way to store a bunch of boolean flags but you want it to end up as human readable code and you don't want to deal with binary operations. Step 1) use an enum for your flags and give the enum the [Flags()] attribute. Now you have a human readable flags, but you still have to do bitwise operations on them to set, unset and get these flags. So, wrap the whole thing in a struct giving a boolean property as accessor for each flag.

struct ByteFlag
{
  // Internal storage of our flags -----------------------------
  [Flags()]
  enum flags : byte
  {
    IsCool = 1,
    IsTasty = 2
  }

  flags bf;
  // Initialize, set and retrieve the underlying storage ------
  public byter(byte pInit)
  {
    bf = (byteFlag)pInit;
  }
  public byte ToByte()
  {
    return (byte)bf;
  }
  public void FromByte(byte pValue)
  {
    bf = (byteFlag)pValue;
  }

  // Accessors ------------------------------------------------
  public bool IsCool
  {
    get { return (bf&byteFlag.IsCool)==byteFlag.IsCool; }
    set { bf = (value)?bf|byteFlag.IsCool:bf&~byteFlag.IsCool; }
  }
  public bool IsTasty
  {
    get { return (bf&byteFlag.IsTasty)==byteFlag.IsTasty; }
    set { bf = (value)?bf|byteFlag.IsTasty:bf&~byteFlag.IsTasty; }
  }
}

Now you can do store that flag in a byte and when you want to manipulate the flags it's as simple as:

ByteFlag byteFlag = new ByteFlag(theByteOfStorage);
byteFlag.IsCool = true;
Console.WriteLine("IsTasty: {0}",byteFlag.IsTasty);
  • in geek
  • 1 min read

Could you please sign this?

Right, so i started thinking I'm doing too many trips, because the client and server could just sign their packages. But that doesn't actually prove anything. After all, someone could have just connected to a server, received their package and then used it to respond to someone connecting to them. That way they'd be pretending to be the other server. The package would be validly signed but not prove the servers identity. Basically we do still need the random phrase challenges from both sides

This does bring up the whole Man-In-The-Middle problem. I could still simply become a proxy between two peers. Not wanting any central service for registering, I'll probably just go with adding the IP in both the client and server packets. That way we can check that the message originates from the same IP we're talking to.

  • in net
  • 2 min read

RSA PK encryption with .NET and Mono

Experimented with System.Security.Cryptography.RSACryptoServiceProvider last night to see if i could use it for Public Key encryption between .NET and Mono. Worked flawlessly.

This goes back to my networking thread. I'm currently contemplating using PKI to establish trust relationships between peers. For the sake of this discussion, there is a serving peer, the one that is connected to, hereafter the server, and a client peer, the one that makes the connection, hereafter the client.

Each peer keeps the public keys of its trusted peers. When a client connects it sends its own public key to identify itself. The server first checks whether it trusts that public key at all, then, having passed that test, encrypts a packed containing a random string, plus its own public key with the client's public key. Only the true owner of the public key can decipher the message.

Now it's the client's turn to check whether it in turn trusts the servers public key. If it trusts the public key, the client encrypts a new packet containing the original random string, plus one of its own divising with the server's public key and sends this back. Again, only the true owner of the public key can decipher the response to its own challenge.

Decrypting the package the server can now verify that the client is truly the owner of the public key, and finally it sends the client's random string back encrypted with the client's public key as acknowledgment of its trust and affirming its own identity to the client.

At this point the client should send all the public keys it trusts to the server and the server responds with the subset it trusts as well . Now we have a relationship that allows the peers to trust each other and know what information they may pass on directly, given the others level of trust and what information it would have to proxy, being the trusted intermediary.

I don't know if I'm re-inventing something obvious here. It seems fairly similar to the SSL handshake, except that I don't care about encrypted communication, and therefore don't use SSL. I just want to establish trust on connect.

  • in net
  • 1 min read

EnvDTE going nowhere

Ok, no answers on my quest on what I can cast Events to. No matter, I can always add better support for Events later. Time i get back to it and start adding "add code" drop downs to the code browser. We'll see how it goes from there