In an earlier post I showed how you can take any Resources object(especially PinnedBufferMemoryStream and UnmanagedMemoryStreamWrapper) and obtain it’s byte[]. Unfortunately on the web this is not sufficient because when the byte[] is sent to a client. The browser expects a mime type header to tell it how to handle the byte[]. A partial solution is to simply map each mnemonic to a specific known mime type. This is a fragile solution.
If you are doing localization then a mnemonic such as SiteLogo may be different mime types in different languages, for example:
- en-US: image is a .bmp
- es-Mx: image is a .png
- fr-CA image is a .gif (animated)
- dk-DK image is a .tiff
Of course, one could simply specify to human-ware that they must all be the same. In reality, if the web-site is a commercial product, this will usually break down.
The practical solution is to use Magic Numbers. Use the magic number to determine the file type, then lookup the mime type based on the file type. Gary Kessler has a good set of magic numbers AKA File signatures to get you started. In a new file type shows up, then just collect a sample and run a utility to obtain additional file signatures.
Capturing Signatures
We create two structures to capture the file signatures:
struct MagicNumberInternal
{
/// <summary>
/// Number of bytes offset
/// </summary>
public Int32 Offset;
/// <summary>
/// The byte sequence expected there
/// </summary>
public Byte[] Offsetbytes;
}
struct MagicNumberSet
{
/// <summary>
/// The bytes expected at the start of the file
/// </summary>
public Byte[] Startbytes;
/// <summary>
/// The bytes expected at the end of the file
/// </summary>
public Byte[] Endbytes;
/// <summary>
/// The bytes expected at some offset in the file
/// </summary>
public MagicNumberInternal[] Offsets;
}
The next item is defining how to input the magic numbers. I opted for a simple string format as shown below:
- AA3G – the leading bytes
- *AA3G – the ending bytes
- AA3G*AA3G – trailing and leading bytes
- *12345:AA3G* – bytes located at an offset of 12345.
- AA3G|FFEEAC – Alternative leading characters
- BBCC*12345:AA3G*987654:AA3G*FFEE – leading, ending and two sets of offset bytes.
This information is parsed easily as shown below:
/// <summary>
/// Character to separate sets
/// </summary>
char[] barSep = { '|' };
/// <summary>
/// Character to separate parts: AA*AA is start end,
/// AA*4563:FF*EE means start with AA, at 4563 bytes FF is found, ends with EE
/// </summary>
char[] pSep = { '*' };
/// <summary>
/// Delimiter between bytes offset (int) and the bytes (Hex)
/// </summary>
char[] offSep = { ':' };
Queue<MagicNumberSet> _MagicNumberPatterns;
ParsePattern
/// <summary>
/// Parses the magic pattern into a structure.
/// </summary>
/// <param name="pattern">A Magic Number pattern, for example AA*4563:FF*EE </param>
/// <returns>An structure with the contents ready for use</returns>
MagicNumberSet ParsePattern(string pattern)
{
var mniQueue = new Queue<MagicNumberInternal>();
MagicNumberSet ret = new MagicNumberSet() { Startbytes = null, Endbytes = null, Offsets= null };
var parts = pattern.Split(pSep, StringSplitOptions.None);
for (var i = 0; i < parts.Length; i++)
{
switch (i)
{
case 0:
if (!String.IsNullOrEmpty(parts[0]))
ret.Startbytes = parts[0].FromHexToByte();
break;
default:
if (!String.IsNullOrEmpty(parts[i]))
{
var breakdown = parts[i].Split(offSep, StringSplitOptions.RemoveEmptyEntries);
switch (breakdown.Length)
{
case 1:
ret.Endbytes = breakdown[0].FromHexToByte();
break;
case 2:
mniQueue.Enqueue(new MagicNumberInternal { Offset = int.Parse(breakdown[0]), Offsetbytes = breakdown[1].FromHexToByte() });
break;
default:
throw new DataMisalignedException(String.Format("The MagicNumber string {0} is invalid", pattern));
}
}
break;
}
}
return ret;
}
The test to see if there is a match is shown below:
IsMatch
/// <summary>
/// Takes some data and see if there are any matches.
/// </summary>
/// <param name="data"></param>
/// <returns></returns>
bool IsMatch(byte[] data)
{
foreach (MagicNumberSet set in _MagicNumberPatterns.ToArray())
{
bool pre = set.Startbytes == null || PrefixMatch(data, set.Startbytes);
bool post = set.Endbytes == null || PostfixMatch(data, set.Endbytes);
bool mid = true;
if (set.Offsets != null)
{
foreach (var item in set.Offsets)
{
if (item.Offset < 1 || item.Offsetbytes == null || OffsetMatch(data, item.Offsetbytes, item.Offset))
{
mid = false;
}
}
}
if (pre && post && mid)
{
return true;
}
}
if (mniQueue.Count > 0)
{
ret.Offsets = mniQueue.ToArray();
}
return false;
}
Thus, the supports multiple signatures for one file type with complex matching patterns. An dictionary of <magicNumberSignature, MimeType> is the closing piece of the solution.
Post Script:
Some of the supporting functions are shown below.
FromHexToByte
/// <summary>
/// Utility to take 0x2DF8 or 2DF8 into a byte array
/// such as { [2D],[F8] }
/// </summary>
/// <param name="value">A string </param>
/// <returns>an equivalent byte arra</returns>
public static byte[] FromHexToByte(this string value)
{
Queue<byte> ret = new Queue<byte>();
var local = value.StartsWith("0x", StringComparison.OrdinalIgnoreCase) ? value.Substring(2) : value;
for (int i = 0; i < local.Length; i = i + 2)
{
ret.Enqueue(Byte.Parse(local.Substring(i, 2), System.Globalization.NumberStyles.HexNumber));
}
return ret.ToArray();
}
PostfixMatch
/// <summary>
/// Determines if the bytes match for the first common bytes
/// </summary>
/// <param name="a">a byte array</param>
/// <param name="b">a byte array</param>
/// <returns>true if all of the bytes are the same</returns>
static bool PostfixMatch(byte[] a, byte[] b)
{
var length = a.Length > b.Length ? b.Length : a.Length;
for (int i = 0; i < length; i++)
{
if (a[a.Length - i] != b[b.Length - i])
{
return false;
}
}
return true;
}
OffsetMatch
/// <summary>
/// Determines if the bytes match for the first common bytes
/// </summary>
/// <param name="a">a byte array</param>
/// <param name="b">a byte array</param>
/// <returns>true if all of the bytes are the same</returns>
static bool OffsetMatch(byte[] a, byte[] b, int offset)
{
if (a.Length < b.Length + offset)
{
return false;
}
for (int i = 0; i < b.Length; i++)
{
if (a[i + offset] != b[i])
{
return false;
}
}
return true;
}
PrefixMatch
/// <summary>
/// Determines if the bytes match at the specified offset
/// </summary>
/// <param name="a">a byte array</param>
/// <param name="b">a byte array</param>
/// <returns>true if all of the bytes are the same</returns>
static bool PrefixMatch(byte[] a, byte[] b)
{
var length = a.Length > b.Length ? b.Length : a.Length;
for (int i = 0; i < length; i++)
{
if (a[i] != b[i])
{
return false;
}
}
return true;
}
Posts
