KEEP IN TOUCH CALL US: 877 513 3118
We are celebrating! 2013 has been an amazing year for Kidozen. We started the year announcing the public availability of the platform, on February, and now we are closing it announcing that we raised 5 Million dollars in Series A funding!!!
Looking back the growing of the platform is astonishing, I’m still amazed with what we have accomplished in this two years, since we started the first drafts of the platform on December 2011. I’m very proud of being part of this team.
Now, looking forward, 2014 will be a great year, our team will grow and we will continue increasing the platform.
We are already hiring, if you want to be part of the future of mobile enterprise, check out the profiles we are looking: http://www.linkedin.com/vsearch/j?keywords=kidozen
ADFS 2.0 supports multiple authentication methods through authentication handlers that are mutually exclusive. If one of the handlers runs, the others don’t. There is no way to implement a fallback logic if one the handlers fail to run or the user was not able to provide the expected credentials, so supporting dual authentication such as username/password and client certificates is sometimes problematic. The following list shows the handlers included out of the box with ADFS 2.0,
|1 2 3 4 5 6 7 8|
Those handlers are for Forms Authentication (Username/Password in an html form), Integrated Windows Authentication, Client Cert Authentication, and finally Http Basic Auth Authentication. The order in this list determines the priority by default unless the priority has been changed in the execution context. The order in the execution context can be changed in multiple ways. For example, when you are doing WS-Federation, the relying party can pass an additional “WAuth” query string parameter with the expected authentication type such as “urn:oasis:names:tc:SAML:1.0:am:password” for Forms Authentication or “urn:ietf:rfc:2246” for Client Cert authentication, which overrides the priority in the list set on the configuration. The same thing can be done for SAML 2.0 by changing the authentication context in the SAML Request message or changing the URL.
However, as I said before, in one of the handlers run, you will not have a chance to run any of the other handlers unless you implement some workaround. For example, you might want to add a checkbox in the html form for Forms authentication to allow the user to authenticate with a client certificate. When the checkbox is clicked, you do an http redirect to ADFS but changing the URL this time to use this new authentication method. If the RP is using WS-Federation, that means a new redirect that includes the WAuth query string variable with the value “urn:ietf:rfc:2246”, or a redirect to “auth/sslclient/” in the case of SAML 2.0.
If you want to avoid this manual step and detect the client certificate automatically, more work is involved, and it is what we are going to discuss next.
|1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32|
|1 2 3 4 5|
|1 2 3 4 5 6 7|
This code is very simple. It simply does a redirect when the cert is present (the callback was called with a “true” value). The redirect in this sample is only valid for SAML 2.0
|1 2 3 4|
This code injects our CertCallBack.js script in the page and it also includes the script located in the virtual directory configured to accept client certs (/CertDetection/Script invokes the ASP.NET Handler basically)
That should be all. If the browser detects a client certificate when the script in the CertDetection virtual directory is resolved, the callback will be executed with a true value, making the form to do a redirect to authenticate the client with a Client Certificate instead. Otherwise, the client will see the Form to enter the username and password. We are adding a new virtual directory for not interferring with the existing authentication methods in the ADFS virtual directory.
The Windows Azure Tools 1.7 introduced a new feature for adding content to the Windows Azure projects called “Role Content Folders”. In some scenarios, you might want to add custom content such as static pages, documentation, configuration files or external binaries for example. This is useful for example if you want to deploy a solution not written in .NET such as java implementation, and you don’t want to mix that with the .NET code in the Role project. The following image shows how the content folders are added in the Azure project.
That content is included in the generated Azure package, and it is deployed in the AppRoot folder when the package is finally published in VM in the cloud.
One of the problem with this feature is that you might want to include content with an structure that changes often or content with thousands of folders/files, which requires some tedious manual work in Visual Studio to keep that content updated in the project.
Good news is that you can use a MSBuild task to inject that custom content automatically when the package is being generated. You have to include a custom Target “BeforeRoleAddContent” right after the declaration of the “Microsoft.WindowsAzure.targets” as it is shown bellow,
|1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30|
The example above injects several content folders in two different roles, “SolrMasterHostWorkerRole” and “SolrSlaveHostWorkerRole”. The “Include” attribute specifies the source folder, and the Destination folder within the AppRoot is specified in the Destination element.
Solr is a robust search platform created by the open source community on top of Apache Lucene. It’s completelly written in java, and uses the Lucene java implementation at is core for full-text indexing and search. In addition, it exposes an http web interface for doing the full text searches and perform management tasks. On other hand, we have SQL Azure, which currently does not support full text searches, so these two services complement very well each other.
As Solr is mainly a java implementation, you only have a few alternatives to run in on Windows Azure. You can deploy it as a worker role together with the java runtime machine, or you can deploy it in a VM. As any solution in the cloud, the state persisted in the worker role or VM goes away when the VMs are replaced or they go down. As Solr persists the indexes in disk, you need to make sure it is stored in a permament storage like Azure Drive or the storage service. If you decide to use a worker role, this requires some additional work to make Solr to store the indexes in Azure drive for example, which is a VHD stored in the storage service that can be mounted by the VM as a local disk. Good news is that MS Open Tech has already done this for us. They have a created a template that deployes Solr with a fail-over configuration(master-slave) in two worker roles, one role for the master node, and another role for the slave node. The slave node replicates from the master node, so in case you lost one of them, you still have the other node available. . In addition, it configures a web role with an MVC application that acts as a admin dashboard for doing basic management stuff. This solution is hosted in Github as part of this project Windows-Azure-Solr. The Github site also provides instructions to get the solution deployed in Windows Azure.
The template that you download from GitHub imports data into Solr by crawling some URLs. That’s part of the data-config.xml file that you can find in the configuration folder of the master and slave nodes (SolrMasterWorkerRole\SolrFiles\data-config.xml and SolrSlaveWorkerRole\SolrFiles\data-config.xml). Solr supports the idea of data importers, which can be used to import data from different sources such as existing web sites, files in disk or even a database.
In this case, we will modify that data-config.xml file to use a data importer that pulls data from an existing SQL Azure instance. The following example shows how this data importer configuration looks like,
|1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22|
First of all, we have defined a dataSource element that points to our SQL Azure instance. This is a Jdbc data source that uses the SQL Server driver and sets the connection string in the url attribute. Secondly, we have defined a document, which specifies one or more entities that are mapped from the SQL Azure database with a select statement. The fields section maps the different fields in the document to the fields returned by the select statements. This document is what Solr stores in the index using Lucene. As you can see, three queries have been defined. The first one with the “query” attribute is used by the data importer when a full import of the complete database is done into Solr. The other two queries are used for supporting a delta or partial import scenario. These two are optional and useful only in scenarios where you have frecuent updates and a lot of data to import, which will make the full import considerably slow.
Since this data importer uses the SQL Server driver for the Jdbc data source, you will have to download that package from the Microsoft website and copying it in the folder where Solr looks for the external libraries (SolrMasterWorkerRole\Solr\dist and SolrSlaveWorkerRole\Solr\dist).
We have defined so far the mapping of a document against one or more tables in the database, but Solr still requires the definition of those fields, which are part of the schema. The schema definition can be found in the schema.xml file (SolrMasterWorkerRole\SolrFiles\v44\schema.xml and SolrSlaveWorkerRole\SolrFiles\v44\schema.xml). The following example shows how the schema is modified to include the fields used by the data importer.
|1 2 3 4 5 6|
This is enough to get Solr configured to pull data from SQL Azure, so we are in conditions to deploy Solr into an Windows Azure subscription also using the tool provided by MS Open Tech. If you have installed all the pre-requisites and followed the instructions in the wiki page of the Github website, the following command should be enough to perform that deployment.
Inst4WA.exe -XmlConfigPath “SolrInstWR_V4.4.xml” -subscription “[subscription name]” -location “[location]” -DomainName “[cloud service name]”
If you are lucky enough to get the command to work in the first instance, it will open a new browser instance when the deployment is complete, and it will also redirect you to the MVC dashboard running in the web role.
You are in conditions now to execute a few commands to import the data into Solr. You will see the location and port where the Solr master and slave nodes are running as part of the home page in the dashboard. For doing a full import into the master role, you have to copy that location and port, and append “/dataimport?command=full-import”. For example, “http://mysamplesolr.cloudapp.net:21000/solr/dataimport?command=full-import”. That will start a full import that runs asynchronously, so you can use this other command to check the status of the import “/dataimport?command=status”. For doing a partial import, you only change the command to this one “/dataimport?command=delta-import”. Once the import is complete, you can do a search to verify that everything looks ok. That can be done with the following command “/select?q=[query]”. For example, “http://mysamplesolr.cloudapp.net:21000/solr/select?q=azure”
So you have Solr indexing all your data now, but what happens with the security ?. This thing is open to the world. Anyone can do anything with your Solr instances as everything is public. There are some ways to secure Solr pages by changing some settings in the web server, which is Jetty by default. However, the Solr documentation recommends to put Solr behind a reverse proxy that filters the requests. There are several reverse proxy implementations for Solr in Github, but I will use a different approach for Windows Azure here. Given that the MS Open Tech template includes a web role for the MVC dashboard, and two worker roles for running the Solr master and slave instances, we can make the Solr instances available in internal endpoints only and use the MVC application as a facade or reverse proxy to forward all the requests to these instances. The only public and visible face will be the MVC application in the web role. All the requests for Solr must go through this MVC application first, which can filter any request that looks malicious or any request that can damage the existing indexes. A simple way to do this is to allow only get operations and filter the rest. This is the approach I’ve been taken and implemented as part of a fork created from the MS OpenTech project in Github. This fork is available here.
A time ago I discussed some serious issues in ASP.NET Web API for testing controllers that were using the Request or UrlHelper instances because those required a valid HttpConfiguration instance. You could initialize the HttpConfiguration instace, but that required a lot of work and some ugly code as part of your tests.
The following method in a controller works to describe the problem,
|1 2 3 4 5 6 7|
The Request propery is being used to generate a content negotiated response, and the Url property to infer the new location for the resource. If you use ASP.NET Web API as it is today, you will have to write a lot of custom code to initialize the configuration instance as it is shown below.
|1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20|
Glenn Block wrote a very nice extension as part of the Web API Contrib project for making this configuration much more simpler. This extension is available here, https://github.com/WebApiContrib/WebAPIContrib/blob/master/src/WebApiContrib.Testing/ApiControllerExtensions.cs.
However, we can left all this in the past with the new ASP.NET Web API vNext release. The introduction of the IHttpActionResult interface (Equivalent to ActionResult in ASP.NET MVC) has simplified a lot the unit testing story for controllers. A controller method can return now an implementation of IHttpActionResult, which internally uses the Request or the UrlHelper for link generation, so the unit test only cares about the returned IHttpActionResult instance.
The following code shows the same controller method using an instance of IHttpActionResult.
|1 2 3 4 5 6 7 8 9 10 11 12|
CreatedAtRouteNegotiatedContentResult is an implementation also included in the framework for handling this scenario. A new resource is created and the location is set in the response message.
The code for the unit test is much simpler too.
|1 2 3 4 5 6 7 8 9|
The UrlHelper class has also been modified to make most of its methods virtual, so they can be mocked or overriden as part of an unit test. You can use this in case you still need to rely on the Url instance in the controller. For example, if you have a controller method like this one.
|1 2 3 4 5 6 7 8 9 10 11 12|
You can create a custom class that derives from UrlHelper and overrides the Link method.
|1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19|
Originally posted on inside.auth0.com.
How we use Puppet, GitHub, TeamCity, Windows Azure, Amazon EC2 and Route53 to ship Auth0
There are two ways companies deploys web applications to the cloud: PaaS and IaaS. With Platform as a Service you usually deploy your applications to the vendor's platform. Infrastructure as a Service is the basic cloud-service model: you get servers, usually vms.
We started auth0 using Heroku's Platform as a Service but soon we decided to provide a self-hosted option of Auth0 for some customers. In addition, we wanted to have the same deployment mechanism for the cloud/public version and the appliance version. So, we decided to move to IaaS.
What I am going to show here is the work after several iterations at a very high level. It is not a silver bullet and you probably don't need this (yet?) but it is another option to consider. Even if this is exactly what you are looking for, the best advice I can give you "don't architect everything from start", this come from the work of several weeks and took several iterations but we never ceased to ship, everything evolved and keeps evolving, and we keep tuning the process a lot on the run.
Picture yourself deploying your application to your brand new vm today. What is the first thing you will do? Well, if you have a node.js application installing node will be a good start. But probably you will need to install 10 other things as well, and changing the famous ulimit, configuring logrotate, ntp and so on. Then you will copy your application somewhere in the disk and configure it as a service and so on. Where do you keep this recipe?
Puppet is a tool for configuration management. Rather than an install script you describe at a high level the state of the resources in the server. When you run puppet it will check everything and then it will do whatever it takes to put the server in that specific state, from removing a file to installing a software.
After you have your configuration in a language like this, deploying to a new server is very easy. Sometimes I modify the configuration via ssh to test and then I update my puppet scripts.
There is another concept emerging called InmutableServers, it is a very interesting way and there seems to be some companies using it.
Auth0 is very modularized and it is not a single web application but a network of less than ten. Every web application is a node application. Our core is a web app without ui which handles the authentication flows and provide a rest interface, dashboard is another web application which is just an interface to our core where you can configure and test most of the settings, docs is another app full of markdown tutorials to name a few.
We use github private repositories because we already had a lot of things opensourced there.
We use branches to develop new features and when it is ready we merge to master. Master is always deployable. We took some of the concepts from a talk we saw; "How Github uses Github to build Github". When something is ready? is a tricky question but we are very responsible and self organized team, we do pull-requests from branch to master when we want the approval of our peers. Teamcity automatically run all tests and will mark the pull requests as OK, this is a very useful feature of TC. But the most important thing we do in this stage are code reviews.
Very often we send a branch to one of our 4 tests environments with our hubot (a personal bot on the chat room):
with that in place we can review a living instance of the branch in a environment similar to production.
Then we iterate until we finally merge.
This is what works for us now, anyone of the team can merge or push directly to master and we consciously decide when we should do the pull-request ceremony.
We used Jenkins for six months but it failed a lot, I had to rebuild few of the plugins we were using. Then I had a short fantasy to build our own CI server but we choose teamcity since I had use it before, I knew how to set it up and it is a good product.
Every application is a project in teamcity, when we push to master teamcity does:
1,2,3 are very common even in non-node.js projects. In the 4th step what we do is to move all "dependencies" to "bundleDependencies" in the package.json by doing this, the
npm pack will contain all the modules already preinstalled. The result of the task is the tgz generated by
This will automatically trigger the next task called "configuration". This taks pull our configuration repository written in puppet and all the puppet sub modules, then it will take the last version of every node project and build one ".tgz" for every "site" we have in puppet. We have several "site" implementations like:
The artifact of the configuration task is a tgz with all puppet modules including auth0 and the site.pp. All the packages are uploaded to Azure Blob storage in this stage.
The next task called "cloud deploy" in the CI pipeline will trigger immediately after the config task, it updates the puppetmaster (currently in the same CI server) and then runs the puppet agent in every node of our load balanced stack via ssh. After it deploys to the first node it does a quick test of the node and if there is something wrong it stop it and it will not deploy to the rest of the nodes. Azure load balancer then will take the node out of rotation until we fix the problem in the next push.
We have a backup environment where we continuously deploy, it is on Amazon and in a different region. It has a clone of our database (max 1h stale). This node node is used in case that azure us-east has an outage or something like that, when this happens Route53 will redirect the traffic to the backup environment. We take high availability seriously, read more here. When running in backup mode, all the settings become read-only, this means that you can't change the properties of an Identity Provider however your users will be able to login to your application which is Auth0 critical mission.
Adding a new server to our infrastructure take very few steps:
Assembling an appliance for a customer is very easy as well, we run an script that install puppetmaster in the vm, download the last config from the blob storage and run it. We use Ubuntu JeOS in this case.
I've to skip a lot of details to make this article concise. I hope you find it useful, if there is something you will like to know regarding this don't hesitate to put your question in a comment.