Using Azure Queues in EPiServer

EPiServer’s DXC cloud based Azure platform as a service provides the availability and scalability necessary to auto scale enterprise level high transaction applications. DXC has bundled within it’ set up an Azure Storage Account used for Blob Storage and a Service Bus used for managing events such as invalidation of cache between instances. Once your application is deployed to DXC, it just works and let’s the partner concentrate on delivering a robust quality build.

An Azure Storage Account comes with the following types of storage but do note that while these are technically within the DXC storage account they are not yet exposed for integration purposes.

However that shouldn’t mean that we can’t use an Azure Storage account to build our applications. Even if these services are not exposed within DXC, you can set up a Storage Account under another Azure subscription as the cost even under a large volume of integrations is tiny: https://azure.microsoft.com/en-us/pricing/details/storage/queues/

Azure Queues

Azure Queue Storage is for storing messages in the cloud to be exchanged between components of systems. Typically a “producing” system will create a message which is to be processed by the “consumer”.

Azure Queue’s are well used in an EPiServer application to decouple the logic around integrating with third party systems from our EPiServer code base. Once solution design is complete we can treat the EPiServer application which pushes the message as a separate code base to the delivery application. This approach has loads of benefits:

Maintainability

If there are issues with an integration, developers no longer have to debug the web application to get to the bottom of it. They can simply check the messages entering the queue, verify the data in the queue and then the effort can instantly focus on the pushing of the message or delivery application components.

Managing both applications in separate deployment pipelines also mean we don’t have to test the web application code base if we are updating an integration and vice versa.

Scalability

Azure Queue’s are insanely scalable coping with up to 20,000 messages per second! All our EPiServer application has to do is push the message.

Error Tracking

When the delivery of a message fails, Azure Queue’s will be default retry 5 times. However both the retry attempts and time between retries can be configured. When a message fails past the set thresholds it enters a “poison” queue where it can be viewed and have additional error handling implemented. I’ll touch on this further down the article.

Quality

In addition to each of the reasons above, having separate application in separate CI deployment pipelines gives us the foundation to write independent sets of robust unit tests. Done well, we will find mistakes in code long before they get deployed a live environment.

Time To Market

After initial solution design is complete, you can have separate independent development teams working on the web and integration applications increasing velocity and hopefully getting us live earlier! Hiring managers also love it as they do not necessarily need new hires to have EPiServer skills to make an impact on a project.

Let’s see some Code!

Putting a message on a queue

In this example we’ll put a very simple Contact on a queue which can will later be synced to an Email Marketing Platform.

First create your storage account on your subscription as documented by Microsoft: https://code.visualstudio.com/tutorials/static-website/create-storage

If you like, you can download Azure Storage Explorer to inspect your Storage account:

https://docs.microsoft.com/en-us/azure/vs-azure-tools-storage-manage-with-storage-explorer?tabs=windows

The first step in our EPiServer application is then to install the following nuget package: https://www.nuget.org/packages/WindowsAzure.Storage/

On events such as a new user registering for your site, you can then simply push the relevant data to a queue and from that point on the web application will have not responsibility for the delivery of the package to the third party platform. Pushing a message to a queue is very simple as illustrated in this sample where i push a json serialised contact model.

azure1

Introducing Azure Functions

Now that we have a message saved in an Azure Queue, we need to process it and send to our third party application.

Azure Functions is an event driven, serverless computing service that extends the existing Azure platform allowing us to implement code triggered by events occurring in Azure. Functions can also be extended to third party service and on-premises systems but in this post we’re focused on Azure.

Using Visual Studio we can simply add a new Azure Functions project to our solution.

azure2

On the set up screen we can choose a Queue Trigger template setting the “ConnectionString” and “Queue name” to the same values as in the previous step.

azure3

This will scaffold a very simple function:

azure4

Every time a message is inserted on our “EmailMarketingPlatformQueue” queue the function will get executed and from here we can implement the integration with our third party system.

Azure Functions Pro Tips

This library gives you Dependency Injection capabilities you will require to use a test driven approach to your development, :

https://github.com/BorisWilhelms/azure-function-dependency-injection

What if it fails, logging retries?

By default, a queue trigger will be tried 5 times before it then inserts the message into a “poison” queue.

You can configure both the number of retry attempts and the time between retries in the host.json file which is added to your solution in the VS set up.

https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-queue#host-json

Alternatively you can set the retry policy programmatically as you push messages to a queue.

If you want to do extra processing when a message enters the poison queue, you can simply write a new Azure Function to be triggered on that event.

Suggestions for DXC

I would love to see this approach supported out of the box in the DXC and these are the two things i would like to see happen:

Support for Deployment of Functions to DXC
Future versions of the EPiServer.Azure package to provide methods to push data to other Azure Storage services

I don’t know if this is on the roadmap for DXC and if it isn’t I trust there are good reasons behind that. However in the meantime, running your own Storage account can repay the small cost over time.

Choosing between Umbraco and EPiServer CMS

EPiServer and Umbraco are excellent .Net CMS platforms.

EPiServer is an enterprise level fully licensed product while Umbraco is an open source option driven by a passionate community of developers.

If you are choosing between these two options, there are a number of factors that you should consider and I want to touch on these without delving too deep into any particular topic.

Both EPiServer and Umbraco come with MVC versions which provide full control of rendered markup. They also support standard CMS features such as publishing pipelines, versioning, personalisation, reusable content blocks and forms. The extent to which these features are polished and customisable varies but the fact is that they are supported by both platforms.

So what are the major factors to consider when choosing between these platforms?

Budget

This is obvious so let’s get it out of the way first.

EPiServer is a very polished enterprise level CMS with a price tag to match. If your client does not have a budget to match the investment required, it is better to look at open source options and this is where you will invariably turn to Umbraco.

Admin UI

Think about the user personas who will be editing and approving content. How technology savvy are they and how intuitive will they expect the editing and administrative interface to be.

The UI of EPiServer provides the content editor with inline page editing. Would such an intuitive feature be viewed as a big win for your target audience?

What are the functional requirements for administration and how much customisation is required? Is there a requirement for advanced content approval workflows? Will the administrator need a strong level of control over the execution of scheduled tasks for example?

The UI of Umbraco is very functional and can be customised through custom page properties. However, it is not as customizable or polished as the EPiServer UI.

Personalisation

Personalisation provides the ability to personalise the content being served to particular users. It is a very powerful marketing tool when used to its potential.

CMS’s administrators can define sets of attributes that a user must fulfil to belong to a particular group of visitors and thus be served personalised content. The possibilities for defining this criterion are endless. It could be related to the profile of the user by attributes such as age or location. It could also be defined by external factors such as the day of the week, time of day or weather if we get more advanced.

There is a Personalisation Groups package for Umbraco which comes with a number of criteria you can use.

However, EPiServer is more powerful in the personalisation space and for me, this is where it starts to stand out from open source competitors. Personalisation comes out of the box and there is an impressive array of criteria included which allows you to create visitor groups for anything from number of site visits to the referrer or search key words.

The interface for defining visitor groups and ability to in page edit the configuration on a given page provide a very intuitive experience for editors.

EPiServer also has a simple programming interface so developers can quickly create custom criteria.

EPiServer is the clear winner on this feature. The question is that if an advanced level of personalisation is a pivotal requirement or if it may be in the future.

Commerce

Is e-Commerce either a requirement now or will it be in the future? If so, what scale of commerce do you need to plan for?

Umbraco has a number of both paid and licensed e-Commerce packages with Merchello being an open source option I have experience with. While there is an initial learning curve from a developer perspective, it does a solid job at providing single market e-Commerce capabilities. It also provides a nice interface for catalogue, customer and sales management. If you need a CMS application with basic commerce, I do recommend checking out the Umbraco and Merchello combination.

EPiServer Commerce is an enterprise commerce add-on and a whole different beast. By enterprise commerce, I mean that it has the power to cope with multi-market solutions where each market is customisable in terms of products, pricing, warehouse integration, ERP integrations, sales, payment providers, tax and all the complexities this level of solution brings to the table.

In essence, these options are competing at totally different sides of the commerce market and it should be quite obvious which end of the market the solution you are scoping is veering to as you delve into your business requirements.

Systems integration

Neither Umbraco or EPiServer come with integrated analytics or CRM. This, in my opinion, is a good thing because I want the flexibility to use the best solutions out there for my requirement rather than use a half-baked implementation that comes bundled with a CMS.

Both products have connectors which are sometimes free and sometimes licensed to connect to your platform of choice. Although it is possible to develop your integrations on both platforms, using to a suitable connector can drastically reduce the cost of development.

If there are custom implementations required on an internal system such as an ERP, the EPiServer service API is developer friendly providing the tools do this efficiently. The Umbraco API is not as powerful so there will be more leg work for the development team which can impact project costs.

Localisation

Both CMS’s have single site multi-lingual localisation capabilities.

EPiServer has very strong localisation support and does not add significantly to the development cost. It’s UI is very clean ensuring that it can be very easily managed by an administrator. On certain projects, this can be a very real cost saver.

If localisation is not a core requirement, Umbraco may well be sufficient. It does require more developer work and does not have the intuitive UI of EPiserver but it may be enough.

Deployments

Think about the roadmap of the application after initial build. How often do you expect there to be maintenance and future development phases carried out?

Largely the deployment process will be quite standard but they do differ on CMS templates. The object which defines a CMS template is called a “document type” in Umbraco and a “content type” in EPiServer but they are the same concept.

Umbraco requires an administrator to manually set up and configure “document types” in the admin UI each time new or updated templates are deployed. This get’s inefficient as you deploy to stage, UAT and live environments. More importantly it adds the inevitability of human error.

EPiServer on the other hand allows the developer to set up “content types” programmatically. As code is deployed, new or updated templates will be automatically configured.

Limiting configuration and user error associated with deployment could be a minor win or it could be a game changer. This really depends on the complexity of the project at hand.

Developer Support

Although Umbraco is a free CMS option, you need to sign up to a subscription if you want developer support from the Umbraco team. Before you buy a subscription, consider that there is a large and very active community of developers who will often be able to provide assistance in resolving issues.

As you would expect with a licensed product, EPiServer provides developer support through it’s service desk and from my experience reasonably fast turn around times.

So which is better?

Both are excellent platforms.

The question is which is better for your client and the project at hand.

Strive to choose the right tool for the right job by assessing the information above and you will be on the right path.

Picking the correct Agile Process

In software development, there are two popular and much-debated categories of process workflow in software development: Waterfall and Agile.

Agile software development requires thorough planning with rigid requirements at development time. Agile software development is executed in short iterations where planning is done in its entirety up front in Waterfall.

Even on a Waterfall project with a fixed scope & cost, a strong management team will move the operational strategy in the direction of agile development borrowing processes and practices to improve team collaboration and efficiency.

Waterfall

Software professionals will cringe when they hear the word “Waterfall”.

The truth is that Waterfall does have its place and the predictability is suited to certain projects.

The problems with a waterfall process arise when a project passes that threshold where it is no longer fully predictable in terms of requirements, workflows and technical challenges.

As a project management discipline, waterfall also makes a lot of sense. In this model a significant amount of time is invested up front in thoroughly defining requirements so that every possible caveat is covered and we know when it is going to be delivered.
The team will then plan how they are going to approach the problem and how long it will take. The project plan contains time for testing and bug fixing before deployment with maintenance being carried out under warranty or retainer.

This is perfect in fields where requirements don’t change after scoping and implementation time is predictable to an extent with the construction industry being a good example.

The beauty and challenge of software projects is that we are doing something unique. The nature of such endeavours is that these undertakings are inherently unpredictable.

The reasons I believe Waterfall isn’t always the correct choice for software are:

1. Estimations are done up front and do not take the status and learnings gained throughout the implementation into account.

2. Waterfall assumes requirements are thoroughly fleshed out on all levels. Particularly on large, complex scopes of work, this is an unrealistic misconception.

3. A fixed scope does not respond to evolving customer needs over time. The finished product can be what a client thought they wanted rather than the solution they really need.

4. Little or no feedback loop to harness learnings and gain efficiencies during implementation.

Agile

There are several subsets of Agile. Each tie back to the principle of iterative work cycles.

Another defining characteristic is flexibility on all sides so that scope evolves to meet the needs of the customer throughout implementation. This can be best achieved by releasing what is a minimum viable product early and assessing its effectiveness against key KPI’s.

The term agile development does not correlate with a particular way of doing things. It is an umbrella term which covers a number of different processes and practices. Two of the most widely adopted processes are Kanban and Scrum.

**Tell me about Kanban?**

Kanban interestingly has its roots in car manufacturing with it originally being the brain child of Toyota in the 1940’s. Toyota took their inspiration from an unlikely source: the supermarket! The supermarket clerks restocked grocery items by their stores inventory levels and not by their vendors supply.

Toyota used this quite simple and yet profound realisation to drive the adoption of a revolutionary “just-in-time” process that would match inventory with demand throughout the entire production lifecycle in turn achieving higher levels of efficiency and throughput.

In software the idea is that a Kanban board will have various columns that represent task statuses e.g. in progress, test, ready to deploy. If you are trying to visualise this, think Trello boards. With each column containing tasks, it gives the entire team a visual representation of the current project status.

Kanban as a software development process emphasises limiting the number of tasks in any column prioritising a smooth, lean flow right through to deployment. The number of tasks in progress at any given time should correlate with the capacity of the team.

If a bottleneck is identified with too many features sitting under any one status, the entire team will work together to identify the cause of the impasse and keep features moving through.

What about Scrum?

Scrum teams develop projects or products in an iterative, incremental model. It borrows its name from the term in Rugby Union which refers to when a team of players interlocked together push in a common direction against the other team.

Development in Scrum is executed in cycles called Sprints which are typically 2 week iterations.
A Sprint planning meeting happens at the start of a sprint where work is prioritised, estimated and committed to depending on capacity. After a sprint planning, there are no changes in the scope of what is being committed to during this iteration.

Each morning the Scrum team will gather to give each other a status update and discuss next steps working towards the common goal of delivering all functionality committed to.

At the end of the sprint the team demonstrates what has been delivered to all stakeholders before entering into a retrospective where an open, honest conversations takes place discussing learnings and what actions are necessary to improve the velocity of the next sprint.

The most important takeaway for this is that scope is frozen during a Sprint. While Scrum does embrace change, any changes need to go into the next sprint.

Kanban or Scrum?

I have seen both Kanban & Scrum used effectively in the right situations. It’s not a case of one methodology being better than the other, it’s about using the right methodology in the right place.

In both Kanban and Scrum, a prioritised and unified product backlog is essential to the success of the project. Product management will prioritise that backlog having a clear vision of what gets worked on first and the development team will decide how.

Another consideration is the deployment infrastructure. Kanban relies on frequent releases of software so a mature continuous deployment infrastructure is essential.

Deployments in Scrum are on the other hand less frequent so manual deployments, while not ideal, would not be as much as an obstacle here. Many Scrum teams choose to batch the product of their sprints into quarterly release plans although there are others who choose to release monthly or in extreme cases by sprint.

Last but definitely not least the single biggest factor to consider when choosing a methodology is the type of project or product this is and what your goals are. If your goal is to frequently add new features to an existing product, Kanban is the way to go. Spotify is the most famous exponent of Kanban done right. A solid product with frequent releases adding new features.

Scrum on the other hand is very well suited to moving new scopes of work forward with product management being very closely integrated with the development team. Velocity in scrum projects is tracked against sprint planning estimations which gives the product management team improved project plan visibility.

Keep it simple

Whatever methodology you choose for your project, the most important thing to realise is that every organisation is different and what works for some will not necessarily work for all.

Remember that the defining characteristic of agile is flexibility.

Try something, see what works, see what doesn’t work, get feedback from the entire team and evolve. The key is to keep it simple. Do not over-complicate and you will reap the benefits.

8 Web Security Standards for every .Net MVC web application

Security is a beast! As a bare minimum, we need to ensure our applications meet industry best practices when released into the wild.

The vast majority of attacks seek to exploit common vulnerabilities so by implementing basic standards we can dramatically reduce risk.

Reviewing the 8 practices below are a great start when assessing the security of your coding practices.

1. Always use an ORM

SQL injection, the injection of untrusted SQL into the database via a query inputted to the application, is still one of the most common vulnerabilities on the web today. The fact this most basic type of attack is still prevalent points to the volume of poorly designed and implemented applications on the web today.

Protecting yourself against SQL Injection is easy and can be done in numerous ways by simply ensuring that data inputted to your application is never directly injected at database level.

Parameterised stored procedures with input validated against a whitelist is an acceptable approach as long as the developer does not concatenate strings within the SP.

However, the best way to protect your application is to use an ORM every time you are querying or writing to the database. There are numerous options for .Net ORM’s with the Entity Framework being the most popular choice but it really is a personal preference.

2. Using Get and Post attributes correctly

Get data is appended to the query string and available publicly while Post data is passed to the server as part of the request body.

Action methods should be decorated with the appropriate Get or Post attribute.

Get should be used when a call is made to a controller function that retrieves data to load a page.


[HttpGet]
 public ActionResult Index(string username)
 {
 // ... etc
 }

Any controller function that accepts and processes data should have the Post attribute.

[HttpPost]
 public ActionResult Delete(string username)
 {
 // ... etc
 }

3. Anti-Forgery Tokens

When the server is processing a request, it should validate that the source of the request to confirm that the user is not the subject of a Cross-Site Request Forgery (CSRF) attack.

MVC provides antiforgery token helpers to that gives you a means to detect and block CSRF attacks through putting a hidden field in the form and then checking that the correct value was submitted on the server side.

To use this first place an HTML.AntiForgeryToken() helper in the form


@using (Html.BeginForm()) {

@Html.AntiForgeryToken()

  <!-- Rest of code here -->
}

This will output a token in the HTML and also give the visitor a corresponding cookie.

Next we need to validate that incoming post at the target action method.

To do this we just need to add the ValidateAntiForgeryToken attribute to the action method.


public ViewResult SubmitPasswordChangeUpdate()
{
 // add code here
}

As a standard all form requests should be validated with anti-forgery tokens. Otherwise you have opened a vulnerability in your application.

4. SSL

SSL will ensure that personal information going up the line is encrypted in the request body.

Ensure your site has a valid SSL certificate. Then force any pages that either retrieve or send personal information from http to https .

Use the .Net RequiresHttps action filter by applying the attribute where required as follows:

[RequireHttps]
public ActionResult SignIn()
{
    return View();
}

Or why not apply the filter globally to force all pages to Https.

protected void Application_Start()
{
    GlobalFilters.Filters.Add(new RequireHttpsAttribute());

    // rest of code here
}

5. Cachable SSL Pages

Any pages that are forced onto HTTPS and contains sensitive data should disable caching. Add “Cache-Control: no-cache, no-store, must-revalidate” to the response headers.

This can be achieved in .Net by setting specific response values. Do this by creating a custom action filter attribute as demonstrated here:

public class NoCacheAttribute : ActionFilterAttribute
{
   public override void OnResultExecuting(ResultExecutingContext filterContext)
   {
      if (filterContext == null)
throw new ArgumentNullException(filterContext);
var cache = GetCache(filterContext);
      cache.SetExpires(DateTime.UtcNow.AddDays(-1));
      cache.SetValidUntilExpires(false);
      cache.SetRevalidation(HttpCacheRevalidation.AllCaches);
      cache.SetCacheability(HttpCacheability.NoCache);
      cache.SetNoStore();
      base.OnResultExecuting(filterContext);
   }
protected virtual HttpCachePolicyBase GetCache(ResultExecutingContext filterContext)
   {
      return filterContext.HttpContext.Response.Cache;
   }
}

6. Password Storage

Ensure you have a strong password policy implemented. As an absolute minimum your policy should insist on 7 character comprising of lowercase, uppercase, numeric and special characters.

You can do that by setting membership provider attributes in the web.config or by configuring the provider to use a regular expression to validate passwords as in this example.

<membership>
     <providers add passwordStrengthRegularExpression= "^(?=.*\d)(?=.*[a-z])(?=.*[A-Z]).{8,10}$" ...></providers>
</membership>

Passwords stored in the database should be hashed using a salt. Better yet you could move to a more robust hashing algorithm than .Net’s standard SHA1.

7. Error Handling and Custom Error Pages

It is imperative that we never give away information to a potential threat about the internal workings of our system.

If a visitor can generate an error message like below it is technically poor and not a good user experience.

All unhandled exceptions should be redirected to a custom error page that only contains a friendly generic message (i.e. message that does not reveal any information nor the type of error that occurred).

This can also be set in the web.config like below:

 <customErrors mode="RemoteOnly" defaultRedirect="Error">
 <error statusCode="404" redirect="Error"" >
 </customErrors>

8 Secure Cookie

These days we live a large amount of our lives online and how do websites know who we are? Through cookies of course!

If there is an XSS vulnerability on the site, a malicious user could inject some JS that could modify cookies on page load or worse still, send the cookies to a remote server! If somebody has your browser cookies they essentially have your identity for that website.

A HTTP only cookie tell’s the browser that this cookie should only be accessed by the server. Any attempt to access the cookie from a client script will be forbidden.

You can add the following line to your application which will set the HttpOnly header on cookies for your application:

<httpCookies httpOnlyCookies="true">

HttpOnly cookies don’t make you immune from XSS attacks but they do raise the bar