Tuesday, January 16, 2018

Curl command for Microsoft Cognitive Services product QnA Maker

This is the curl command to send QnA Maker a question and receive the answer:

curl -H "Content-Type: application/json" -H "Ocp-Apim-Subscription-Key:<subscriptionKey>" -X POST -d '{"question":"<question>"}' https://westus.api.cognitive.microsoft.com/qnamaker/v2.0/knowledgebases/<appID>/generateAnswer

<subscriptionKey> found on https://www.qnamaker.ai/UserSettings
<appID> found in url when view app, ?kbid=<appID>
<question> question such as "How do I ...?"

Monday, January 15, 2018

Debug mocha.js with VSCode

This is my cheat sheet for debugging the mocha.js project from VSCode while adding fixes to it. This is not about how to debug ANY project that is using mocha for testing.

The mocha.js repository has a build system that builds into /bin/mocha.

I don't use a package.json script. I just use a launch.json mocha test configuration:

{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Mocha Tests",
"program": "${workspaceFolder}/bin/_mocha",
"args": [
"--colors",
"${workspaceFolder}/test/reporters/dina.spec.js"
],
"internalConsoleOptions": "openOnSessionStart"
}
]
}


  • The program value has to be changed from the node_modules directory to the /bin directory. The top level file is supposed to be bin/_mocha but it won't debug. 
  • Remove any timeout args from the args array. 
  • Specify the exact file you want to debug into such as "dina.spec.js"

Sunday, October 15, 2017

Paperspace.com for Windows 10 Desktops in the cloud

I recently needed a clean Windows 10 desktop for testing purposes. I couldn't use Docker, Vagrant, VirtualBox or any of the other local-to-my-computer VM solutions. I tried the Amazon Workspaces solution but it had the typical UX of 30 questions I don't know the answer to. I just wanted a desktop that could download from the Internet.

Paperspace.com to the rescue!

Within 5 minutes I had a Windows 10 desktop in a browser and the hardest questions where how much CPU and ram I wanted. The cost was incredibly inexpensive too.

The test only took 10 minutes and I destroyed the cloud machine after that.

This was the easiest, fastest, and cheapest solution to the problem.

Monday, September 18, 2017

SQL Server AAD Authentication Error

Are you seeing this error from your ADO.NET Code Trying to Connect To Your SQL Server via Active Directory Authentication:

Exception=System.Data.SqlClient.SqlException (0x80131904): Failed to authenticate the user NT Authority\Anonymous Logon in Active Directory (Authentication=ActiveDirectoryIntegrated). Error code 0x800703FA; state 10 Illegal operation attempted on a registry key that has been marked for deletion. 

The issue might be that you are running a windows service or a scheduled task with a user that has logged off. In Windows Server the registry hive for the current user is loaded when the user logs into the machine, and unloaded when they log off. This means that if you service account is running under a user that is logged off then the registry hive will not be available. ADO.NET AAD in .NET 4.6.1 uses the registry hive of the running user -- which could be the user of a service.

To solve this problem you need to tell Windows Server not to unload the registry hive for users when they log off.

{6230289B-5BEE-409e-932A-2F01FA407A92}


The Asp.net Core 2.0 Environment for NodeJs Developers

NodeJs & .Net Core

NodeJs and the Asp.net ecosystems have used very different paradigms until recently. This article will show you where they are similar – the 10,000 feet view – when using the Asp.net Core 2.0 environment.

With NodeJs, as the developer you might have chosen to build a web server or a cli tool.  Your favorite interactive development environment (IDE) might just be a terminal window or Google Chrome DevTools.

With Asp.net (not .Net Core), Microsoft provided the web server (IIS) and the IDE (Visual Studio). You can develop many different applications including web sites and cli tools. With .Net Core 2.0, the focus is on portable code, and the result feels much closer to NodeJs.

For this article, I'm going to ignore all issues that would make the comparison apples to oranges and instead focus on issues that make the comparison apples to apples.

Moving forward, any reference to Asp.net Core 2.0 will be just .Net Core. NodeJs 6.9.1 will be just Node. The example project name will be kittens so you can see how it is used when it is relevant.

Text Only

The files of both .Net Core and Node are both text files. A project has a single configuration file. The top of each code file references the libraries that are used in the file.

In .Net Core 1.0, the project file was a package.json, but in .Net Core 2.0, the project file is a *.csproj file. The package file for .Net Core is created with the cli of "dotnet new...". The XML of the .Net Core file will be familiar to .Net developers.

<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.0</TargetFramework>
<RuntimeIdentifiers>win10-x64;osx.10.11-x64;linux-x64;ubuntu.14.04-x64</RuntimeIdentifiers>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.All" Version="2.0.0" />
</ItemGroup>
</Project>

In Node, the project file is the package.json created by the NPM cli of "npm init" with the result of a JSON object containing information about the project. I added expressjs to the package so the dependencies object would have an entry.

{
  "name": "kittens",
  "version": "1.0.0",
  "description": "",
  "main": "kittens.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "express": "^4.15.4"
  }
}

In their own way, each of the package files configure the project but in very different ways. The .Net Core file is focused on the .Net Core version and final built assets of the project – things the compiler and dependencies need to run.

The Node package.json file lists all dependencies to pull in, as well as meta data about the project – it is the single location a developer touches to control the project as a whole, including building and testing. I usually have a long list of items in the Node package.json scripts section for building, testing, and running.

Execution 

Both Node and .Net Core applications are built via terminal/command line. Both are executed in the context of the parent application: dotnet or node.

While Node needs a single point of entry of a javascript file, such as kitten.js, .Net Core uses the project name (<name>.csproj).

> node kittens.js

> dotnet kittens.dll

Dependencies are added to Node projects with the npm or yarn package managers. The dependencies are placed in a folder named node_modules.

> npm add <packagename>

For .Net Core dependencies, use the NuGet package manager. There isn't a single list of dependencies in a human readable format that is opened/viewed by developers on a regular basis.

> dotnet add package <packagename>

Both npm and dotnet allow you to specify versions but it is optional. Npm and yarn have been dealing with package interdepencies and version conflicts. I believe both are now able to avoid version conflicts of interdepencies.

Deployment 

Both platforms allow you to 'git clone' then install the package dependencies and run.

.Net Core uses a folder containing many files depending on the execution platform (windows, mac, linux).  For Node, just install and run a single file. Node will then find the rest of the files.

> dotnet restore && dotnet ./<path-to-bin-directory>/kittens.dll

> npm install && node kittens.js

Serverless 

The reason I mention serverless is you may not want to deal with the execution environment details.
Node has a variety of host platforms and open-source projects to run serverless.

Currently, the best way to run .Net Core serverless right now is Azure Functions. While there are other platforms such as AWS Lamdba that can run .Net Core code, those platforms may be slightly behind (or vastly behind) release .Net Core platform ability.

Docker 

Docker containers are a great way to develop and deploy applications. I always look for owner created containers. If it works for them, it will work for me.

Here are the owner created containers for each:

node
microsoft/aspnetcore

For Node, the specific owner container isn't that interesting because there are a million ways you may want to run Node and a different container may get you much closer to your needs in terms of dependencies and tools.

For .Net Core, the owner container is probably the best bet for the short-term while the project is still new-ish.


Sunday, May 7, 2017

Agile Manifesto -- Some 16 years later

"In 2001, 17 individuals gathered in the Wasatch mountains of Utah to find common ground around Agile. After much skiing, talking, relaxing, and eating, they arrived at four common values that led to the development of the Agile Manifesto."[source] [Wikipedia]

Going to AgileManifecto.org, we see just 14 people listed, what appears to be their feeling on what promised so much hope some 15 years ago.



  1. Mike Beedle 
  2. Arie van Bennekum
  3. Alistair Cockburn
    1. " “Agile has become overly decorated. Let’s scrape away those decorations for a minute, and get back to the center of agile. The center of agile is … ” [2015]
  4. Ward Cunningham
    1. " "I have seen my ideas diluted as they diffused through the industry". You said "I’d much rather move to the next idea than struggle to keep the last idea pure." [2011]
  5. Martin Fowler
  6. Jim Highsmith - 
    1. Don’t “Control” Agile Projects
    2. Agile Bureaucracy: When Practices become Principles
  7. Andrew Hunt The Failure of Agile (2015)
  8. Ron Jeffries
    1. Do you want Crappy Agile? [2016] " it encourages you to track “metrics”. Do you want Crappy Agile? That’s how you get Crappy Agile, at least far too often."
    2. "Scrum is good, when done as intended. Otherwise it can be oppressive and dangerous to developers. Let's study: Defense Against the Dark Arts of Scrum." Dark Scrum
  9. Jon Kern
    1. "The only thing that has me mildly torn--and only mildly, because "to each his own" and "what-evs" come to mind--is the craze surrounding Scrum. Scrum, scrum, scrum, scrum. Part of me is happy that there are legions out there turning the sod under, trying to sow the agile seeds via Scrum. Another part of me thinks it is like Corn-to-Ethanol. A waste of energy for the dumbfounded among us, spurred on by lots of lobbying." [2011]
    2. Have not posted on his blog about agile since 2011
  10. Brian Marick "In his keynote at the Agile Development Practices conference, Brian Marick described values missing from the Agile Manifesto. His view is that the Manifesto was essentially a marketing document, aimed at getting business to give agile a chance." [2008]
    1. "As Agile moves into bigger companies and into less adventurous ones, those undocumented values are getting watered down. If that continues, I’m afraid Agile will be this decade’s fad that changes nothing."
  11. Robert C. Martin (aka Uncle Bob) moved on to "Clean Code" 
  12. Ken Schwaber owns Scrum.org
  13. Jeff Sutherland CEO of Scrum Inc
  14. Dave Thomas 

    Agile is Dead (Long Live Agility) - PragDave

Missing are: [source]

Question - Agile is good if selling it is your bread and butter

Many agile signatures appear to have become disillusioned by how Agile is commonly being adapted in the industry. Some people have stopped writing and posting on it. Others have proposed derivatives of it in the hope of getting back to core values. Others are making a good living from preaching the gospel of agile.

My view?  Simple, I started using RAD back in 1979 and been using the concepts (without the dogma) for the last 37 years.  The best implementation have been where the product owner/customer was very technical and detail orientated and put in the needed hours. The worst cases were scrum was magically expected to make management easy and not require their heavy engagement or hours  with the products they owned or manage.

I like the concepts and practice them instinctively, I hate the "rote-application of the manifesto" 

Monday, May 1, 2017

Running Tests in Docker for Front-end Developers (ng2+)

Unit tests (karma/jasmine) & End-to-end tests (protractor)

If you are comfortable with Docker and Angular 2, all you need is the Dockerfile. If you are new to testing in Docker, this short article will bring you up to speed. If you need more help getting up to speed with Docker, begin with my previous article: Docker for Angular 2 devs.

Installation 

In order to develop Angular 2+ in a container, you need to install Docker on a computer or VM. Docker takes a bit of memory and can take a lot of space so the biggest box you can give it is best. Once Docker is installed, start is up.

Make sure all your docker commands are run from the folder that contains the Dockerfile.

Dockerfile 

Docker works by reading the description of a Dockerfile (or several in conjunction), to build out an image. Once the image is running, it is called a container.

This particular Dockerfile is based on a Docker image that already has headless chrome, markadams/chromium-xvfb-js:7. There are several Dockerfiles at Docker hub for headless chrome. If this base image or my additions do not work, feel free to play around with other Dockerfiles. You will find references/websites at the bottom of this article.

What Does this Dockerfile do 

This Dockerfile installs the angular2-webpack-starter (and all dependencies). I use it as a development environment that I connect to and run the tests myself as I develop. I'm using the angular2-webpack-starter as my base test that the Dockerfile has all the needed dependencies to complete both the unit tests and the end-to-end tests successfully.

If the starter can run both types of tests, it should be fairly straightforward to add your own repository to the container for development and testing. The container has the full stack for this: Git, NPM, Yarn, Node, and Angular+, ng-cli, Webpack, Karma, Jasmine, Protractor, http-server.

Why use the angular2-webpack-starter?

I wanted an Angular 2+ project that used as much of the same stack as my current projects that also had unit tests and e2e tests. In order to validate I have the right Dockerfile configuration, all I have to do is run both types of tests in this repository. If they succeed, my own tests for my own projects should succeed as well.

Can I see it working?  

Yes, I have a short 4 minute video showing it from building the docker image to the final e2e test run. I've cut out 11 minutes of Docker building the image as there is a lot of installation of dependencies.



Using as a Development Environment 

I use this container as my main development & testing environment. I share my local repository folder into the container with the –v param and expose a small range of ports to allow for several front-end and api servers to run at once.

The angular2-webpack-starter runs on port 3000 so I open 3000-3005. This allows you to use your host computer to access the website (http://localhost:3000) as long as the port isn't in use on your host machine.

What are the important parts of the Dockerfile?  

There are definitely packages and settings in the Dockerfile you may not use, but there are a few you need to keep.

Protractor needs to know where the Chrome bin directory is. If you remove that environment variable, Protractor will not be able to run the tests. Protractor also needs the default-jre.

Getting karma/jasmine tests working in a container using headless chrome seems to be easy compared to protractor e2e tests. If you change the file, always use the protractor e2e test run as the verification that the change did no harm.

Git Commit Hash 

Since the repositories for both my Dockerfile and the angular2-webpack-starter will change over time, I'm noting the git commit hashes of the versions I used.

https://github.com/AngularClass/angular2-webpack-starter
commit e9521a42  - Sat Apr 22 19:27:33 2017 -0400

https://github.com/dfberry/DockerFiles/blob/master/headless-chromium/Dockerfile
commit 38b7554c Sun Apr 30 13:23:17 2017 -0700

References 

https://github.com/mark-adams/docker-chromium-xvfb
https://hub.docker.com/r/yukinying/chrome-headless/
https://hub.docker.com/r/justinribeiro/chrome-headless/