Friday, March 25, 2011

Segmenting the Queue across Multiple Instances in Windows Azure

A common usage pattern in cloud computing is to process all tasks that are similar in the same instance, at the same time leveraging the scalability of cloud computing by running multiple instances for all the tasks. Grouping like tasks within an instance is done for a couple of reasons:

  1. Avoid data contention across tenants. For example, if you are aggregating data you might want to performance all the aggregation in memory before writing the data to storage. If you can segment the data that need to be aggregated together on a single tenant you will not have another tenant beating you to the write.
  2. Optimize your local memory cache. If you are caching expensive to retrieve data in the instance, segmenting your tasks to a instance with the task appropriate cache can increase your cache hit ratio. In other words different cached data specific to the work that the instance is doing.

This article will talk about how to successfully accomplish task segmentation in Windows Azure.

The solution that I am proposing works like this:

  1. Add all your tasks onto a Windows Azure Queue
  2. Create multiple instances of a worker role that knows how to process the tasks
  3. Each worker role:
    1. Takes 32 messages from the queue for 60 seconds.
    2. Segment out the messages that it will process and process them
    3. Deletes only the messages that it processed.

60 seconds is an arbitrary number, it needs to be adjusted based on your work load.

Adding Tasks to the Queue

I add tasks to the queue to guarantee that they will be completed. The worker role:

  1. Gets the task from the queue in the form of a queue message.
  2. Processes it.
  3. Then deletes the task from the queue.

If the worker role is terminated for any reason, the task will appear back on the queue when its visibility window expires. Because the queue is acting as the transaction manager, worker roles can: terminate, reboot, have exceptions, and run out of memory without loss of data. The trick is that the worker role has to be able to reprocess a previous processed partially completed task successfully. For more about this concept, see Steve Marx’s Blog post entitled: Deleting Windows Azure Queue Messages: Handling Exceptions.

Serialization of an Object onto the Queue

One thing I do to make life easy on myself is that I create C# class that support serialization, and then write them to the queue as bytes. This is a handy way to store the data I need for processing. An example of the object might look like this:

[Serializable]
public class Task
{
    public string Name { get; set; }
}

Here are the methods I use to serialize and deserialize the object. Notice I am gzipping it to reduce its size, Windows Azure Queues can only holds maximum size of eight kilobytes, so if you have bigger classes then the example, gzipping can help you get a little more data in the queue.

public T DeserializeCompressed<T>(byte[] input) where T : class
{
    using (var ms = new MemoryStream())
    {
        var binaryFormatter = new BinaryFormatter();
        ms.Write(input, 0, input.Length);
        ms.Seek(0, SeekOrigin.Begin);
        ms.Position = 0;

        using (var cs = new GZipStream(ms, CompressionMode.Decompress, true))
        {
            var deserializedRaw = binaryFormatter.Deserialize(cs);
            cs.Close();
            return deserializedRaw as T;
        }
    }
}

public static byte[] SerializeUncompressed<T>(T input) where T : class
{
    using (var ms = new MemoryStream())
    {
        var binaryFormatter = new BinaryFormatter();
        ms.Position = 0;
        ms.Seek(0, SeekOrigin.Begin);

        binaryFormatter.Serialize(ms, input);
        ms.Close();
        return ms.ToArray();
    }
}

Segmenting the Messages

The technique segment the messages is to hash a value (in the example below it is the task name) that I want to use for segmentation. Then, I determine the number of running instances for this worker role and divide it. If I take the modulo and compare it to the instance id of the current instance equality means this task name should be processed on this instance, otherwise it is for another instance

The code for the math above looks like this:

 

/// <summary>
/// Hashes the Object And Determines If This Instance Should Be Processing It
/// </summary>
/// <param name="obj">Object To Hash</param>
/// <returns>True If This Instance Should Process It, Otherwise False</returns>
public static bool ShouldProcess(object obj)
{
    int hash = obj.GetHashCode();

    return ((hash % WorkerRole.InstanceCount) == WorkerRole.InstanceId);
}

 

The GetHashCode() method is support by all objects in the CLR. You can override it if you want to create your own hash for your object.

Getting the instance identifier as an integer for the running instance is tricky in Windows Azure:

/// <summary>
/// Cached Instance Id Of The Current Windows Azure Instance
/// </summary>
private static int? instanceId = null;

/// <summary>
/// Get the Current Windows Azure Instance Id
/// </summary>
public static int InstanceId
{
    get
    {
        if (instanceId == null)
        {
            // WWB: Array of Instance Ids (as strings) Sorted By Name
            string[] array = RoleEnvironment.CurrentRoleInstance.Role.Instances.Select(_ => _.Id).ToArray();
            Array.Sort(array);

            // WWB: Find the One That Matches The Current Id From the List
            for (int index = 0; index < array.Length; index++)
            {
                if (array[index] == RoleEnvironment.CurrentRoleInstance.Id)
                {
                    // WWB: Cache The Index
                    instanceId = index;
                    break;
                }
            }
        }

        return instanceId.Value;
    }
}

I also need to get the instance count to divide the hash by. That code looks like this:

/// <summary>
/// Count of Windows Azure Instances In This Role.
/// </summary>
public static int InstanceCount
{
    get
    {
        return RoleEnvironment.CurrentRoleInstance.Role.Instances.Count;
    }
}

Putting It All Together

Now that we have seen the pieces of the puzzle, let’s put it all together to process the tasks on the queue. Here is what the code looks like:

// WWB: Get 100 Message For 60 Seconds
IEnumerable<CloudQueueMessage> messages = cloudQueue.GetMessages(32, TimeSpan.FromSeconds(60));

// WWB: Iterate Across All the Messages
foreach(CloudQueueMessage message in messages)
{
    // WWB: Get A Task Object From the Cloud Message
    Task task = this.DeserializeCompressed<Task>(message.AsBytes);

    // WWB: Segment the Tasks Based On Name
    if (WorkerRole.ShouldProcess(task.Name))
    {
        this.Process(task);

        cloudQueue.DeleteMessage(message);
    }
}

 

You will have to adjust the visibility time of the messages from 60 seconds to whatever works for you, the maximum time to process the maximum number of messages which is 32.

 

Notice that the code only deletes messages that have been processed. If an exception is thrown, the message will not get deleted, and the worker role will be recycled by Windows Azure. Once the worker role is restarted, it will pick up the message again and try to process.

 

So what happens to the messages that are not deleted – those whose names are not for this instance? They are made visible to the other instances after their visibility window timeouts out – 60 seconds in the code example above. This allows the other instances to pick them up and process them. There is no code that has to be implemented to make this happen, it is in the design of Windows Azure Queues.

Why Here Why Now?

Why does the example split the tasks after they are read from the queue? Why not before they are inserted? The code could create a bunch of queues, and then segment the tasks when they are written to the queue – with each queue having similar tasks. The reason I do it this way is that I can easily increase the number of instances in Windows Azure using the Windows Azure Management Portal, scaling down and up depending on the amount of work I need to accomplish. It is much harder to increase or decrease the number of queues.

Summary

The code demonstrated above will localize processing to a particular Windows Azure instance based on property of the task that needs to be accomplished as it an elegant solution to segmentation in Windows Azure.

 

{6230289B-5BEE-409e-932A-2F01FA407A92}

Thursday, March 24, 2011

SQL Server: Rounding DateTime to Intervals

There are times when you are wanting to produce bar charts that you wish to round times to the quarter hour and keep it as a time. The typical solution is to cast as a varchar() and use the string, unfortunately this works well only for some intervals. Doing this properly is not hard, but you need to be careful not to have numeric calculation artifacts creeping in. Below, I am trying to round everything to the closest quarter hour, other periods just require some change of numbers.

  • 15 minutes –> 24 x 4 =96 Intervals
  • 10 minutes –> 24 x 6 = 144 Intervals
  • 1 minute –> 24 x 60 =1440 Intervals.

 

The Oops not perfect solution

The function below seems to create the needed result – until you test it and discover that the times are not always precise (arithmetic approximation  issues). This can cause problems with some usages.

 

ALTER FUNCTION QtrHr ( @When DateTime ) RETURNS DateTime AS BEGIN DECLARE @est numeric(38,30) Set @est=Round(Cast(@When as numeric(38,30)) * 96.0,0)/96.0 SET @When= Cast(@est as DateTime) RETURN @When END GO select dbo.QtrHr(GetDate()),GetDate()

 

image

A more reliable approach

I rewrote the function to eliminate reliance on mathematical functions being precise. I grab the day part and then the number of 15 minute intervals, combining them with the DateAdd function. Everything now works fine.

ALTER FUNCTION QtrHr ( @When DateTime ) RETURNS DateTime AS BEGIN DECLARE @est numeric(38,30) Declare @day int Declare @min int Set @est=Round(Cast(@When as numeric(38,30)) * 96.0,0) SET @Day=@est/96.0 Set @Min=(@est % 96)*15 SET @When=DateAdd(mi,@min,Cast(@Day as DateTime)) RETURN @When END

image

WP7 Pivot’s TitleTemplate contains Title & Subtitle

I tried to add to Textblocks to the Pivot':TitleTemplate and got an error "The property 'VisualTree' is set more than once." In order to work around this, I had to add a containing object (Stackpanel) that could contain both objects.

In order to do that, create a TitleTemplate that contains a StackPanel with your Title and Subtitle Textblocks inside. The image below shows the Title and Subtitle above the pivot header.

image

The XAML to produce this is below:
   1:              <controls:Pivot.TitleTemplate>
   2:                  <DataTemplate>
   3:                      <StackPanel>
   4:                      <TextBlock x:Name="TextBlockPivotDeploymentItemHeading"
   5:                          Text="Title"
   6:                          FontSize="20"
   7:                          TextWrapping="Wrap"/>
   8:                      <TextBlock x:Name="subtitle"
   9:                          Text="Subtitle"
  10:                          Foreground="Gray"
  11:                          TextWrapping="Wrap"/> 
  12:                          </StackPanel>                               
  13:                  </DataTemplate>
  14:              </controls:Pivot.TitleTemplate>

Monday, March 14, 2011

IT Hiring: Time to reread Huxley’s Brave New World?

Actually for many managers today, read it for the first time. (It’s on line here)

 

“It's an absurdity. An Alpha-decanted, Alpha-conditioned man would go mad if he had to do Epsilon Semi-Moron work–go mad, or start smashing things up. Alphas can be completely socialized–but only on condition that you make them do Alpha work. Only an Epsilon can be expected to make Epsilon sacrifices, for the good reason that for him they aren't sacrifices; they're the line of least resistance. His conditioning has laid down rails along which he's got to run. He can't help himself; he's foredoomed.”

For the illiterate, Alpha’s are the  smartest ones. The Harvard Business Review just made a video available called “Hiring: Finding People Who Fit” which echo similar thoughts.

 

Recently I have been mentoring several younger folks, and for some, I see the definite attitude that “All people should be made in my image”. No ability (or even consideration) of walking in someone else’s shoes.

 

When I have managed groups (up to 150 bodies) in the past, I have traditionally look to understand each person, their ability, motivation and skill levels and then see where they are the best fit. Often I have encountered people that are in a very satisfying job for family/friends expectations on them, but they are unsuited for the job (yes, they do it satisfactorily and will until they break through their adequate skill ceiling and proceed to crash). These folks are not happy to do this job, but less unhappy then bucking pressure on them. Those folks I attempt to encourage self-realization and hope they will eventually proceed to a career that is both a skill/ability match and self-satisfying.  My attitude is simple, you work for each person’s best interest and the side-effect is that they will work for your best interest (and the groups).

 

Take a look at the video and see if it has applies to experiences you have had. If you are doing a startup, this can be critical for the startup success.

Friday, March 11, 2011

Client Web Sites, Licensing and breach of contract

Yesterday I spent a hour and a half with a local developer who develops websites for local firms. He called because of my earlier posts on copyright issues. After getting a night to sleep on it, I suddenly said “dung, it’s a legal mine field today”. I will scenario two cases:

  • A site using some form of open license software (i.e. JQUERY etc)
  • A site using some 3rd party component that the developer is licensed for (for example, FLASH, TELERIK etc)

“Our contract says that the customer owns the code”

Without a lot of clean legal qualification of what the exactly means – you are in breach of contract with both of the above scenarios! The customer would need to sign the dozen of pages of legalesse required to qualify this. Why????

  • If you are giving them ownership of the code and using JQUERY, then you are saying that they own JQUERY and can legally sue anyone else that uses the JQUERY.
  • If you exclude 3rd party components but supply code that sets or alter properties of these components, you are effectively granting a developer license to the customer.
    • If the customer purchases the software, then it is better – provided the customer is the purchaser of record and after you finish it, the “Developer licensee” of record is transfer back to the customer.
    • IDEALLY, the developer would have their own license – and the client license would never be temporarily transferred to the developer. It keeps matter clean.

The courts will interpret any contract or note written on an invoice based on what it says and how the common man would interpret it: not what is intended or understood by technical types.

“Our contract says that this is a work for hire”

First, this means that you need to make sure that any UI/graphic designer that you subcontracted work to is done as a “work for hire” and include a copy of their signed contract with papers deliver to the customer. A simple invoice with this scribbled on it leaves things unclear (without both parties signatures on the document – it’s not a contract!).

The United StatesCopyright Act of 1976 states:

A "work made for hire" is— (1) a work prepared by an employee within the scope of his or her employment; or (2) a work specially ordered or commissioned for use as a contribution to a collective work, as a part of a motion picture or other audiovisual work, as a translation, as a supplementary work, as a compilation, as an instructional text, as a test, as answer material for a test, or as an atlas, if the parties expressly agree in a written instrument signed by them that the work shall be considered a work made for hire. (17 U.S.C. § 101)

This can potentially cause problems because any libraries of code that your firm has written in the past that is included in the deliverable is now OWNED by the customer and you have no legal right to use it. In fact, if you used it on a different site, you would be obligated to remove the code from those sites – because that usage would breach the copyright that is assigned to the customer by it being a work for hire.

 

For graphic elements there is an additional problem, if the UI person use stock images (they may be legally licensed to do so), they cannot legally grant a work for hire status on items that they do not own a copyright for. This applies to both photography and computer generated images. It can also extend down to the font faces being used in logos and images.

 

“All licenses for you to use the site have been obtained”

The key wording here is “use” – this means that you have not said that they can legally modify the site. Technically, if they want the background image of a page changed, they have not been granted the right to do so. The ideal situation of locking in a customer for simple modifications – if they want to go elsewhere then they need to have a new site developed. Other variation of phrasing could be:

  • “Development includes all licensing from 3rd party for you to legally use the site developed”

The term “modify” should not be used without extensive qualification. It would be good practice to explicity state something like “the right to modify the site is not granted. Any future modifications would require the customer or the customer agent to obtain all of the appropriate licenses”. Focus on the licenses issue – so it is clear that you are not the source of pain, the licenses are.

How do I phrase it to give the customer the code?

Get a lawyer, write a 50 page legal contract, have headers in every file identifying who owns which element of code. The contract will be full of lists of items that are excluded, for example. “ownership does not cover:

  • Fonts used on the site. Including but not exclusive: fonts referenced in code, fonts appearing on images
  • Images used on the site. Modifications of any image is not permitted
  • 3rd party open source elements including:
    • JQUERY
      • list of every JS file used on the site
  • 3rd party licensed components
    • TELERIK
      • list of every dll etc
  • Files that you have been granted an unlimited use for:
    • Headers read  “….”
      • list of every file

Etc etc etc

 

BE VERY CAREFUL of granting an “unrestricted license” because this means that your code libraries could be legally incorporated into the next web developer’s tool kit, in fact, they could start selling a tool set containing it!

 

Protecting yourself against Sales Person wanting to close a sale

Often a sales person would be asked by a customer for something like “do I own the site” and that may be cited on a bid or invoice.  Draft a written policy indicating that no person is authorize to assign or grant anything except a usage license to any one, and get the signature of every sales person on it.

 

Now, it the sales person does it – send them back to the customer to get it removed. If it goes unnoticed and a legal situation arises, you may have some legal protection --- the sales person needs to be immediately fired, and the defense is simple: the sales person committed fraud – having the signed policy indicates that they were not acting in ignorance etc.

 

As always – NOT LEGAL ADVICE – ALWAYS CONSULT A LAWYER

Thursday, March 10, 2011

Litigation: People in LLC Houses…

Lately I have been attempting to assist two former business partners in resolving a dispute. The problem arose because one of them resigned in writing from a LLC leaving the LLC completely in the hands of the other. There can be many reasons for someone doing so:

  • The LLC is getting into murky legal waters and the person wishes to wash their hands of it to avoid being tainted with potential future problems;
  • The LLC is financially underwater (which I believe is correct in this case) and the person decides that future efforts of getting it viable is not worthwhile the effort
    • Typically, the person bailing out will form a new LLC and may pursue a similar business model.

The dispute is an interesting one because it is akin to an employee quitting a company and then turning around and demanding severance, their chair, their PC and all of the software on it that was purchased by the corporation. It appears to be a combination of a sense of over-entitlement and not understanding (or accepting) the consequences of quitting.  On the other side is the still existing corporation that is financially underwater being asked to give up assets. If the corporation does this and then files for bankruptcy(or is forced into it), the remaining manager is between a rock and a hard place to defend such an action against creditors.

 

There are rumors that assets of the corporation were actually removed by the resigning partner after the resignation which opens up another hornet’s nest of legal issues. The issues appear akin to an employee quitting and then claiming a corporate PC loaded with corporate software and proprietary information because of their years of service.

 

Unfortunately, the folks are looking like they will end up in litigation (which may result in bankruptcy of the LLC leaving just losers).

 

Bottom line: If you are a manager of an LLC there are some clean ways of leaving:

  • Ask the other partner to buy you out (if the LLC is not underwater)
  • Demand that the LLC be dissolved and any remaining assets distributed.
    • If underwater, put it into bankruptcy to insure a fair distribution of assets.
  • Resign and make sure you:
    • Return every piece of equipment ever issued.
    • Return all paper work and notes.
    • Delete all records and files of the LLC from all personal machines.
    • Walk away and do not look back – don’t be Lot’s Wife!
      • If the firm is bought for $10,000,000 on the next day – you are out of luck.

Not legal advise – always consult a lawyer.

Silverlight WP7 Change TextBlock Foreground in Item class code

Introduction
This post is a result of research I did for my own app project. I needed to change the Foreground color of a TextBlock via code based on the value of a property at runtime. The app is a Windows Phone 7 Pivot app with the default MVVM classes (provided by VS2010 during project creation). I found plenty of explanation and some sample code but couldn't make it work in my own app. I found the WinMilk app source code on CodePlex and knew that app did something very much like what I wanted to do. Thanks to the app authors for making the code available.

Feature Description
The data concept was of a collection of Items, where each item has a status and a color associated with that status. The visual concept was a PivotItem with a ListBox. The ListBox is bound to the data collection. The ListBox's StackPanel contains a TextBlock for the Item's Name. The Item's Name needs to change color based on the Item status.

Since the object is from a collection, it wasn't as easy as setting the object's Foreground color in the code-behind class. I needed to have the Item class contain the color that was associated with Item's status.

Steps to Complete
1) Create new Visual Studio Project for Windows Phone Pivot Application


2) Edit \ViewModels\ItemViewModel.cs for new property of LineColor

        public string LineColor
        {
            get
            {
                switch (LineOne)
                {
                    //case "design one":
                    //    return "#EA5200";
                    case "runtime on":
                        return "#0060BF";
                    case "runtime two":
                        return "#359AFF";
                    default:
                        return "#EA5200";
                }
            }
        }


The value of the LineOne property determines the LineColor. Notice there is no special markup for property declaration and that return type is string (not Colors, Color, or PaintColorBrush). 


3) Edit the same file, for the LineOne property, adding a NotifyPropertyChanged("LineColor"):

        public string LineOne
        {
            get
            {
                return _lineOne;
            }
            set
            {
                if (value != _lineOne)
                {
                    _lineOne = value;
                    NotifyPropertyChanged("LineOne");
                    NotifyPropertyChanged("LineColor");
                }
            }
        }
The NotifyPropertyChanged code is already provided as part of the project created by Visual Studio. 


4) Change \MainPage.xaml, find the markup for the LineOne Property to bind to the LineColor Property. The   markup provided by the project looked like:



<TextBlock Text="{Binding LineOne}" TextWrapping="Wrap" Foreground="{Binding LineColor}"/>

The image below shows the color change from the first item's LineOne to the second item's LineOne.






Wednesday, March 9, 2011

Entrepreneurs Coffee: Personal Kanban and Lean Startup Methodology

I attended Entrepreneurs Coffee this morning, see a lot of familiar faces and a few new ones. The speaker was Jim Benson, the author of the newly released book "Personal Kanban" as one of the founders of the Seattle's Lean Coffee.

Personal Kanban: Mapping Work | Navigating Life

A few interesting notes that I took are:

  • Jim and his partner(in DC) or clients will by on Skype 100% of the working day, not having a conversation – rather just hearing each other clicking away. When an issue arose, there is no need to make a telephone call or start a Skype conversation… he just speaks up!  Elegant approach. Sharing a virtual skype cubicle.
  • Economy of scale does not work for Knowledge Workers – the more of them that you get involved, the lower the return per employee. I have seen this often and have been known to say “Don’t give me two developers to help me meet the deadline unless you really want to miss the deadline!”. Solo-coding (for me) often results in the highest output. Passing information and making sure it is understood is more time consuming than doing it. On the flip side, if the code is to be maintained, then getting someone else up to speed is essential. The key item is that it is a false assumption that tossing more bodies at a problem in a crisis is what you should do.
  • SCRUM has evolved, originally it was intended to increase communications and break tasks into smaller easier to manage pieces. What has happened is that the process has become formalized. Think of it this way, “there is the spirit of SCRUM and the letter of SCRUM”. SCRUM has become more and more bureaucratic and formal. I saw this 8 years ago at Microsoft where the SCRUM Certified person focused solely on getting the paper-work of SCRUM done (covering you’re a**) and killed communications and team ethos.
    • SCRUM is good conceptually – BEWARE of born-again bureaucrats who are SCRUM-Certified!
  • A book called Visual Meetings was recommended by the U of WW person attending. He made the key points:
    • 90% of typical people are VISUAL LEARNERS
    • 95% of academics are word orientated (likely applies to a lot of software developers and technical writers) too
    • For my other blog, I will likely shift more and more to visual learning styles.
  • Visual Meetings: How Graphics, Sticky Notes and Idea Mapping Can Transform Group Productivity

Those are the main highlights. Join the Entrepreneurs Coffee Meetup group to get notified of the next one!

Tuesday, March 8, 2011

Entity Framework Parameter Differences

When using EF there is no constructor on DbParameter that takes arguments, and that makes sense. It is meant to be overridden by the concrete type under it, usually SqlDbParameter. Because of this there is also no method for AddWithValue on the DbParameterCollection, which confuses people on occasion.

 

I just posted a handy extension method to MSDN forums to help turn this:

 

   1: var p = cmd.CreateParameter();
   2: p.ParameterName = "Latitude";
   3: p.Value = (double?)this.latitude;
   4: cmd.Parameters.Add(p);

 

Into this:

 

   1: cmd.AddParameterWithValue("Latitude", (double?)this.latitude);

 

Here is the method:

   1: /// <summary>
   2: /// Adds a parameter to the command.
   3: /// </summary>
   4: /// <param name="command">
   5: /// The command.
   6: /// </param>
   7: /// <param name="parameterName">
   8: /// Name of the parameter.
   9: /// </param>
  10: /// <param name="parameterValue">
  11: /// The parameter value.
  12: /// </param>
  13: /// <remarks>
  14: /// </remarks>
  15: public static void AddParameterWithValue(this DbCommand command, string parameterName, object parameterValue)
  16: {
  17:     var parameter = command.CreateParameter();
  18:     parameter.ParameterName = parameterName;
  19:     parameter.Value = parameterValue;
  20:     command.Parameters.Add(parameter);
  21: }

Sunday, March 6, 2011

VisualStateManager.GoToState not working?

I’ve had issues with this on and off for a while and thought I should note my solution quickly.

 

Expression Blend will actually use ExtendedVisualStateManager, but the base class VisualStateManager has this solution as well. Basically, due to a bug that was supposed to be fixed, but for whatever reason still exists, the visual state sometimes won’t change using this method. The GoToState method takes a control, a state name string, and Boolean whether to use transition animations.

 

The solution is simple. There is another method called GotoElementState that takes a FrameworkElement, and the rest. Just use this instead and your problems go away.

 

   1: // Avoid this.
   2: VisualStateManager.GoToState(ctrl, stateName, true);
   3:  
   4: // Use this.
   5: VisualStateManager.GoToElementState(ctrl, stateName, true);

Copyright, Open Source, Clean Rooms and Ignorance of the Law

Recently I was reminded (by being an observer to some drama) of the legal constraints that developers should know but frequently do not know – especially the new self-taught developers. I suspect that even some Computer Science graduates are ignorant here.

Monkey See, Monkey Do –Lawyers Knock!

Jack develops websites and a customer points him to a website that he wants emulated. Jack goes to the site and copy portions of the code from the site. Often the code is nothing more than a JavaScript function or a chunk of CSS. He uses these code fragments exactly as written (no renaming variables, changing line orders etc). He brings in a graphic designer that does a brilliant original design.

 

The site is ready, the customer is happy, Jack drops a check in his pocket.   Two months later, the customer phones Jack – he has just received a letter from a lawyer to take down the web site because it contains copyrighted material.

Jack talks the customer into leaving it up. The site provider gets a similar letter 4 weeks later and dutifully takes down the site. The customer phone Jack, angry that his site is down and the site provider refuse to restore it. Jack moves the site to another provider… and the cycle repeats – except this time the customer does not receive a polite letter but either:

  • Papers indicating he is being sued.
  • Knock on the door by the FBI (under 18 U.S.C. Section 1030 or equivalent)

Things go worst….

 

What is the problem? Simple – items on the internet retain copyright unless licensed or placed explicitly in the public domain. There is no “fair use” of code. The rights problem is actually made more severe with open source – because there are hundreds of contributors, there is a significant chance that one of them did a copying of someone’s code off a website. You have an right to use the open source code – but it does not indemnify you if the code violates a copyright.

 

The person in procession of the violation is always accountable! Someone selling bogus DVD or Viagra is not excused because he did not know it was illegal. The best that they can hope for is a deal if they drop their suppliers into the mess.

Clean Rooms

Old timers know about “clean-rooms” from the days of MSDOS. The name has been re-used. The old meaning was putting developers in isolation to develop a product and protect the firm from lawsuits; for example, because a developer reverse engineered code or behavior.

 

In the internet world it would mean that you can do screen shots of a site and create an equivalent NEVER NEVER looking at a single line of source (HTML, JScript, CSS). If the site has “patents” or “patent pending”, then it is even harder because of patents on things like ‘One Click’, so you cannot even do screen shots.

 

The problem is simple, a license, items placed explicitly into the public domain or a truly original creation is needed.

 

Taking your own code with you to the next firm…

If you are an employee of a firm and wrote this awesome library of routines and then leave the firm I have some bad news for you. You cannot use a single item from that library unless you get explicit written permission from your old firm. You created it; they OWN it. You can recreate the library (literally rewrite the library) and likely will improve it in the process.  Every routine etc must be revised. I usually recommend defining a NEW different naming convention, changing the preferred coding patterns and doing a complete refactor with some improvement in at least 50% of the functions.

 

Consultants have more code re-use freedom

Consultants arrive with libraries of code they have evolved from other projects. The use of this code is often viewed as an unrestricted licensing of the code to the customer without the acquisition of ownership. If the contract specifies “work for hire”, then the consultant should clearly mark what is new and novel in this work, and what is simply an unrestricted license.  If the consultant writes work for hire code that he like, he  is still free to refactor the code and add it to his own repository of code – after doing a clean and complete refactor – ideally with some further improvements.

 

If you are responsible for the management of code….

If your crew produces websites – you have an obligation of due-diligence to make sure they do not violate copyright or perform theft of code (a criminal offense). If you do not have the skills, you need to get someone with the skills to come in and:

  • Present and document that your employees have been instructed on best practices at least once a year.
  • Review some code (often this actually amount to testing the dev if they understand the code they are using or just copied it from elsewhere).
  • Make sure that you have insurance against this (I would recommend at least $1,000,000 policy)

If you do not care…

Several years ago, my wife got involved with VRML and creates a short example, 420 bytes of code, that she used on her website. She put a copyright in it.  One day when she Altavista-ed (pre-google) her name she found it on an Israeli site. We contacted a lawyer in Israel, arranged a contingency agreement and told him to go for it. Two months later we received a check. The offender opted to settle for $1200 for the 420 bytes of code instead of our lawyer going the distance for copyright infringement.

 

Many lawyers have spare cycles and a clean copyright lawsuit on contingency can be an easy sale…  We did it internationally with little hassle.

 

Is it fair? Should copyrights and patent be abolished? I have no comment of those issues – they exists and can result in significant financial pain; they are to be respected.

 

THIS IS NOT LEGAL ADVISE – ALWAYS CONSULT WITH YOUR LAWYER ON LEGAL MATTERS

Thursday, March 3, 2011

FYI: I am doing a series of posts for Enterprise Applications on Microsoft’s SQL Server

These will generally be SQL Server focused and are available at:

Some topics covered are how much RAM per core is ideal. How to secure SQL Server better – for example how to time-out SQL Server Sessions after 15 minutes of being idled.

 

Check out the posts and pass them around!

Tuesday, March 1, 2011

Full Employment and Freedom

Last night I watched a retrospective on Alistair Cooke, the classic presenter on PBS (Mystery etc) and one of the signs on a civil rights era protests caught my eye, I remember only the end of it “.. + full employment = freedom”.

 

Looking at history, the truth of this becomes obvious. The end of the Serf system in Europe did not happen by a peasant revolt, but due to the Black Death. It resulted in a labor shortage so serfs could exit their positions and earn income from a land owner whose own serfs were decimated.  High unemployment have been a constant motivation for many revolutions. One of the pro-forma argument in the Communist versus US brand of Freedom days was:

 

“In the Soviet Union we have freedom from hunger, lack of medical care, homelessness, etc. In the US you have freedom to exploit your neighbor, freedom from social contracts between employer and employee, etc.”

So the question becomes, if you are destitute and unemployed with 30% unemployment, with loved ones needing medical care .. are you truly free?  Somewhere in the last 50 years, we may have lost the older understanding of freedom and replaced it with a concept of freedom that may actually be more suitable to medieval Europe.

 

A landed noble of that time would likely argue that there is freedom. If a serf does not wish to work for him, they are free to leave his land with the cloths on their back (all of their animals, tools, etc are owned by the Lord). He does not pay taxes (usually) unless there is a national crisis or a bad king. The question of democratic representation is very amusing: “Look, my lands and title is a closely-held private corporation. Even people in the USA in 2011, agrees that workers do not have the right  to elect board members for a closely-held private corporation. The question of democratic elections makes no sense!”

 

It is interesting to note that no American Citizen may be landed nobles – at the time of the American Revolution the only thing equivalent to a modern multi-national corporation was these closely-held private corporations known then as Noble Families. The founding fathers despise them because they disempowered the person. (Un?)fortunately, in the 1820’s the US introduced corporations (as we now know them) to the world – just like a millennium before, common folks with the needed skills (fighting) rose to position of entitlement (Knighthoods and above). In this case, spare cash invested in shares were the skills that resulted in entitlement.

 

Today, we see across the Arab world protests and revolts – one of the key items has been unemployment. The cry for freedom has gone up with the cry for jobs! Perhaps it is time to redefine a natural right as being the right to influence and have a say in anything that impacts them – which traditionally includes taxes, schools etc. The question that remains unclear is whether it should include their source of employment?