Project 31-A

Many people remember the days when you could use a USB memory stick to boost the performance of Windows. This memory caused me to ask the question: Is there a potential cost saving with little performance impact by going sparse on physical memory and configuring a paging file. For windows folks: For Linux, see TechTalk Joe post . Note his " I/O requests on instance storage does not incur a cost. Only EBS volumes have I/O request charges." so it is not recommended to do if you are running with EBS only. This approach is particularly significant when you are "just over" one the offering levels.  For some configurations, you will not get a CPU boost - by using the paging files. I know recent experience with a commercial SAAS actually had high memory usage but very log CPU (3-5%, even during peak times!). Having 1/2 or even 1/4 the CPUs would not peg the CPU. The question then becomes whether the Paging File on a SSD drive would significantly drop per...

Database sharding has been with us for many years. The concept of cloud instance sharding has not been discussed much. There is a significant financial incentive to do. Consider a component that provides address and/or postal code validation around the world. For the sake of illustration, let us consider 10 regions that each have the same volume of data. Initial tests found that it took 25 GB of data to load all of them in memory. Working of AWS EC2 price list , we find that a m4.2xlarge is needed to run it, at $0.479/hr. This gives us 8 CPUs. If we run with 10 @ 2.5 GB instead, we end up with 10 t2.medium, each with 2 CPU and a cost of $0.052/hr, or $0.52/hr -- which on first impression is more expensive, except we have 20 CPUs instead of 8 CPU. We may have better performance. If one of these instances is a hot spot (like US addresses), then we may end up with  9 instances that each support one region each and perhaps 5 instances supporting the US. As a single instance model...

I have heard many companies complain about how expensive the cloud is becoming as they moved from development to production systems. In theory, the saved costs of greatly reduced staffing of Site Reliability Engineers and reduced hardware costs should compensate -- key word is should.  In reality, this reduction never happens because they are needed to support other systems that will not be migrated for years. There is actually another problem, the architecture is not designed for the pricing model. In the last few years there have been many changes in the application environment, and I suspect many current architectures are locked into past system design patterns. To understand my proposal better, we need to look at the patterns thru the decades. The starting point is the classic client server: Many Clients - one Server - one database server (possibly many databases) As application volume grew, we ended up with multiple servers to handle multiple clients but retaining a s...

One of my current projects is getting data out of Jira into a DataMart to allow fast (and easy) analysis. A library such as TechTalk.JiraRestClient  provides a basic foundation but there is a nasty gotcha. Jira can be heavily customized, often with different projects having dozen of different and unique custom fields. So how can you do one size fits all? You could go down path of modifying the above code to enumerate all of the custom fields (and then have continuous work keeping them in sync) or try something like what I do below: exploiting that JSON and XML are interchangeable and XML in a SQL Server database can actually be real sweet to use. Modifying JiraRestClient The first step requires downloading the code from GitHub and modifying it. In JiraClient.cs method   EnumerateIssuesByQueryInternal add the following code.                 var issues = data.issues ?? Enumerable .Empty< Issue >();       ...