Skip to content

2005

DIY Serialization

Forgot about putting this up as promised. Of course, this is a very simplistic example with only one variable sized field. If you had more, you'd have to also serialize those field's sizes and reconstruction becomes a bit more complicated.

But the important part about this, is really the whole concept of pinning some memory down so that you can manipulate it directly and then releasing it back to the control of the Garbage collector. Pretty cool, really.

[StructLayout(LayoutKind.Sequential)]
struct DIYSerialize
{
  public Int32 Id;
  public byte[] Data;

  public DIYSerialize(Int32 id, string data)
  {
    this.Id = id;
    this.Data = Encoding.ASCII.GetBytes(data);
  }
  public DIYSerialize(byte[] Raw)
  {
    Int32 size = Raw.Length;
    IntPtr pnt = Marshal.AllocHGlobal(size);
    GCHandle pin = GCHandle.Alloc(pnt,GCHandleType.Pinned);
    Marshal.Copy(Raw,0,pnt,size);
    this = (DIYSerialize)Marshal.PtrToStructure(pnt,typeof(DIYSerialize));
    pin.Free();
  }
  public byte[] Serialize()
  {
    Int32 size = Marshal.SizeOf(Id.GetType()) + Data.Length;
    IntPtr pnt = Marshal.AllocHGlobal(size);
    GCHandle pin = GCHandle.Alloc(pnt);
    Marshal.StructureToPtr(this,pnt,false);
    byte[] d = new byte[size];
    Marshal.Copy(pnt,d,0,Data.Length);
    pin.Free();
    Marshal.FreeHGlobal(pnt);
    return d;
  }
}

Interop vs. Serialize

I was this close to just using ISerializable for by binary representation for networking. But then talking to n8, i decided that leaving the door open for java interop was important enough in Enterprise computing, that I couldn't ignore it.

So, I'm back to coming up with my packet structure in manner that i can easily serialize by hand. Right now the plan is something along the line of a fixed sized header followed by a variable sized gzip-Xml chunk and a variable sized raw data chunk. Both of the variable chunks can be 0 length. The header is used for expressing the purpose of the packet, and describing the offsets of the variable packets. The gzip-Xml is for data that the recipient is supposed to parse and act on, while the raw data packet will be just that, raw data that i sued to construct a complete file.

I need to dig up the test code i did for doing manual serialization in .NET. Really, just what would be done in C with a struct and memcpy, but of course, since in C# you don't have pointers and the memory is managed for you, it's a bit more of a hoop jumping exercise. The code is on another machine, so i'll leave that for another post.

Fun with enum bitflags

I've previously learned about the enum [Flags()] attribute, but had forgotten usage and reference to it isn't the best, so i figured, iput this down before it slips my mind again.

Say you want a compact way to store a bunch of boolean flags but you want it to end up as human readable code and you don't want to deal with binary operations. Step 1) use an enum for your flags and give the enum the [Flags()] attribute. Now you have a human readable flags, but you still have to do bitwise operations on them to set, unset and get these flags. So, wrap the whole thing in a struct giving a boolean property as accessor for each flag.

struct ByteFlag
{
  // Internal storage of our flags -----------------------------
  [Flags()]
  enum flags : byte
  {
    IsCool = 1,
    IsTasty = 2
  }

  flags bf;
  // Initialize, set and retrieve the underlying storage ------
  public byter(byte pInit)
  {
    bf = (byteFlag)pInit;
  }
  public byte ToByte()
  {
    return (byte)bf;
  }
  public void FromByte(byte pValue)
  {
    bf = (byteFlag)pValue;
  }

  // Accessors ------------------------------------------------
  public bool IsCool
  {
    get { return (bf&byteFlag.IsCool)==byteFlag.IsCool; }
    set { bf = (value)?bf|byteFlag.IsCool:bf&~byteFlag.IsCool; }
  }
  public bool IsTasty
  {
    get { return (bf&byteFlag.IsTasty)==byteFlag.IsTasty; }
    set { bf = (value)?bf|byteFlag.IsTasty:bf&~byteFlag.IsTasty; }
  }
}

Now you can do store that flag in a byte and when you want to manipulate the flags it's as simple as:

ByteFlag byteFlag = new ByteFlag(theByteOfStorage);
byteFlag.IsCool = true;
Console.WriteLine("IsTasty: {0}",byteFlag.IsTasty);

Could you please sign this?

Right, so i started thinking I'm doing too many trips, because the client and server could just sign their packages. But that doesn't actually prove anything. After all, someone could have just connected to a server, received their package and then used it to respond to someone connecting to them. That way they'd be pretending to be the other server. The package would be validly signed but not prove the servers identity. Basically we do still need the random phrase challenges from both sides

This does bring up the whole Man-In-The-Middle problem. I could still simply become a proxy between two peers. Not wanting any central service for registering, I'll probably just go with adding the IP in both the client and server packets. That way we can check that the message originates from the same IP we're talking to.

RSA PK encryption with .NET and Mono

Experimented with System.Security.Cryptography.RSACryptoServiceProvider last night to see if i could use it for Public Key encryption between .NET and Mono. Worked flawlessly.

This goes back to my networking thread. I'm currently contemplating using PKI to establish trust relationships between peers. For the sake of this discussion, there is a serving peer, the one that is connected to, hereafter the server, and a client peer, the one that makes the connection, hereafter the client.

Each peer keeps the public keys of its trusted peers. When a client connects it sends its own public key to identify itself. The server first checks whether it trusts that public key at all, then, having passed that test, encrypts a packed containing a random string, plus its own public key with the client's public key. Only the true owner of the public key can decipher the message.

Now it's the client's turn to check whether it in turn trusts the servers public key. If it trusts the public key, the client encrypts a new packet containing the original random string, plus one of its own divising with the server's public key and sends this back. Again, only the true owner of the public key can decipher the response to its own challenge.

Decrypting the package the server can now verify that the client is truly the owner of the public key, and finally it sends the client's random string back encrypted with the client's public key as acknowledgment of its trust and affirming its own identity to the client.

At this point the client should send all the public keys it trusts to the server and the server responds with the subset it trusts as well . Now we have a relationship that allows the peers to trust each other and know what information they may pass on directly, given the others level of trust and what information it would have to proxy, being the trusted intermediary.

I don't know if I'm re-inventing something obvious here. It seems fairly similar to the SSL handshake, except that I don't care about encrypted communication, and therefore don't use SSL. I just want to establish trust on connect.

EnvDTE going nowhere

Ok, no answers on my quest on what I can cast Events to. No matter, I can always add better support for Events later. Time i get back to it and start adding "add code" drop downs to the code browser. We'll see how it goes from there

MacMini, the next set-top box?

Ok, not quite programming, but close enough.

I was looking at the MacMini and were I not only invested in using my chipped Xbox with XMBC, I'd seriously consider a MacMini as my Media Center. I mean it's perfect:

  • Has DVI out, easily connectable to my HDTV for high res output
  • Has a DVD player and a DVD drive
  • Is tiny
  • Is ultra quiet
  • Runs iTunes, and all things XMBC considered, iTunes is still the best music player
  • Has BlueTooth for all sorts of remote control options, from Mouse/Keyboard to PamOS to Phone
  • Easily customized via Applescript or perl or even just writing an app in C++, Java, C# (Mono)
  • Easily networked
  • Affordable

Ho hum. But my money is heavily invested in other hobbies already, so this one will have to wait until i can rationalize replace

Gzip under .NET

The kind folks over at ICSharpCode have created a Zlib library with Gzip, Zip, Tar and Bzip2 implementations.

Tried it out last night and it works beautifully. It fits into the networking thread of last in that I've decided that for communication i'm going to use Gzipped Base64 encoded XML. That gives me a nice compact format that's quite portable. Already tried out sending a such encoded message from a perl program on Linux to .NET on Windows and it worked beautifully. And it's 90% smaller than plain old XML.

Maybe if the Fast Infoset Project takes off, there'll be a more compact binary XML. In the meantime, this approach is at least easy for anyone to read, what with the ubiquity of gzip and b64.

Remoting and Mono

After my TCP toying I decided to do some Remoting experiments last night. One of my goals with the network programming i'm doing right now is that the business logic of the code should be portable between .NET and Mono. I know Remoting is a probably not the answer there, since it may not even stay compatible between .NET revisions.

Regardless, I tried it out and it worked great between Linux and Windows. Nice, simple elegant (with a fair share of black magic under the hood). The only pitfall I came across was that if you are using RemotingConfiguration.RegisterActivatedClientType(), you gotta make sure that the type you are using comes from the same assembly on the two platforms. I had a bunch of problems because I compiled the Windows and Linux versions differently at first, but then i just copied the Assembly that contained the classes to be remoted to the linux side and compiled against it. That did the trick.

Now, is there something inbetween Remoting and writing my own protocol with TcpClient for doing Client/Server communication? More research for another night.