Design a monitoring strategy

Operations Manager is the primary monitoring component within System Center which integrates with other Azure services. On-premise SCOM deployments can integrate with Azure through deployment of the Azure Management Pack for Operations Manager.

The ideal configuration, particularly when a larger number of systems are deployed on Azure is to deploy a SCOM Gateway server. Systems can be configured as proxy agents to monitor other systems which are not directly accessible. TCP port 5723 is used for monitoring traffic.

By uploading the SCOM server Public Key to Azure Storage it is possibly to collect data from the Azure Monitor and Diagnostic Service (Windows, Azure, App Sources, Counters, Events, Logs, Dumps).

Azure build in monitoring capabilities include:

  • Default metrics (CPU usage, disk read / write, network in / out)
  • Web apps metrics (CPU time, data in / out, HTTP server errors, requests)
  • Application diagnostics and logging:
    • Application Logging (File System) – access from FTP share for web app
    • Application Logging (Table Storage) – access from specified field table
    • Application Logging (Blob Storage) – access from blob container

Configure application logging, certificate authentication and verbose logging for additional logging and statistics.

3rd party tools are also available often through the marketplace:

Global Service Monitor is an Azure cloud service which works with SCOM to monitors web applications from an end-user perspective and can help identify issues with DNS, network connectivity, etc.

Application Insights provides deep insight into your applications running on a VM or web role and integrates with SCOM to provide a single consolidated view.

When designing application resiliency consider the use of:

  • Availability Sets to split instances across Fault and Update Domains
  • 2 Fault Domains exist in Azure
  • Up to 20 Update (or Upgrade) Domains can be used but only 5 are by default
  • Use Virtual IP (VIP) Swap to test application upgrades

Evaluate hybrid and Azure-hosted architectures for Microsoft System Center deployment

The System Center suite includes the following tools:

  • Operations Manager – monitoring and alerting, integrates with OMS
  • Configuration Manager – Azure integration through Cloud Distribution Point
  • Virtual Machine Manager – hypervisor management (Hyper-V, ESX, XenServer)
  • Orchestrator – automate SC tools and Azure with runbooks
  • Data Protection Manager –  backup to tape, disk or Azure using Online Backup Agent
  • App Controller – provisioning and self service, Public Cloud Connector for Azure integration
  • Service Manager – DBMS platform for self-service, portal and ITIL features
  • Endpoint Protection – security solution for anti-malware

The PowerShell Deployment Toolkit greatly simplifies and accelerates the System Center deployment process.

System authentication can be performed using domain authentication or computer certificates where machines are not domain joined. Consider bandwidth and latency in a hybrid deployment and the optimal placement of services.

 

Design websites for business continuity

Scale-out web apps on Basic, Standard and Premium tiers to 3, 10 and 20 instances respectively. Scale-up to provide additional resource to each instance with higher tiers increasing the limit. Azure SQL also offers Elastic Scale to scale horizontally (sharding) and vertically (250GB Standard, 500GB Premium).

CDN allows content to be replicated globally to be served up to users faster. Files are cached in local Azure Blob storage and accessed using the CDN address. This does not need to be considered when designing web applications which may refer to the original address.

Traffic Manager enables websites to be deployed to multiple regions and accessed using a single URL.

SQL Sync is a feature of Azure SQL which sychronises a database or select tables and columns on a schedule from the source to a HUB database which could be in Azure or on-premise. SQL Sync does not synchronise transactions.

SQL geo-replication allows recovery of a database to another Azure region. Standard tier provides an offline secondary, Premium tier provides up to 4 readable backups.

Web Apps allow a single backup of websites per day, Premium allows up to 50 backups per day. Web App backups will save Web App files, associated database, log files from the website and WebJobs. With Free and Basic FTP(S) or a GIT Clone URL can be used as an alternative.

When designing the data tier it is important to consider the data storage type and scaling of the data:

  • Use of relational (RDBMS) e.g. SQL or non-relational database (e.g. Azure Table Storage, DocumentDB)
  • Scale, size of database supported
  • Normalization of data vs. performance of too many linked tables
  • Backup of historic data to reduce security exposure and database size

Deploy websites

Several methods exist for deploying Azure websites: 

Azure site extensions can be deployed to add custom administrator functionality to your website.Site Control Manager can be setup to manage sites and extensions.

Web deployment packages can be created in Visual Studio and allow quick deployment of websites:

  • .zip file contains all files for deployment including:
    • .cmd file to customise IIS installation
    • .xml files to specify site parameters

Azure App Service Plan provides a mechanism to group web apps and other app services together to manage and scale. Deployment slots allow code to be staged and easily promoted from development to production and backed out. Database connection strings can be sticky per slot to ensure when new code is promoted into production it will use the production database and vice versa.

Web Apps can use a local GIT repository on Azure or an existing source control system. Dropbox can also be used with Azure as an external deployment source with the rollback feature enabled to revert to previous code versions.

 

 

Design web apps for scalability and performance

Websites can be scaled globally by serving content to clients using the CDN service and Azure Traffic Manager for performance load balancing.

Create website using Visual Studio and Azure SDK. Publish web applications using:

  • Azure PowerShell script
  • Publish from Visual Studio using Web Deploy
  • Publish using FTP

Debug published websites using:

  • Monitoring metrics through Azure portal
  • Azure Application Insights
  • Visual Studio for remote debugging
  • Site Control Manager (Project Kudu)

Azure provides support for developing applications and websites in a number of languages:

  • .NET (C#, Visual Basic)
  • Java (tomcat or jetty)
  • Node.js (server-side version of JavaScript)
  • PHP
  • Python

It is possible to run web applications on Virtual Machines, Cloud Services or Web Apps. Each provides benefits, a VM will allow greater flexibility in providing full control of the Operating System and installed applications while a Web App can be a more cost effective option with less management overhead and is easier to scale.

azurevmcsapp

 

Integrate Azure services in a solution

There are a significant number of Azure services available to design your application. Each service offers its own SLAs.

With the growth of semi structured and unstructured data sets alternatives to relational databases such as SQL have become important. Some examples of requirements and Azure services which can be deployed to address them are as follows:

  • Search and query options – DocumentDB, Azure Search
  • Caching – Azure Redis Cache, CDN
  • Recommendation – Azure Search, Azure Machine Learning

Analysing large sets of data in motion has led to the release of many Open Source solutions including Hadoop, Kafka and Storm. Azure offers a range of Big Data and IoT services.

Components of a Big Data solution are likely to include:

 

 

Enterprise mobile applications address the growth of mobile and BYOD. Ingredients of a mobile application are likely to include:

  • Authentication & Identity (Azure AD & Azure AD Authentication Library (ADAL))
  • Access to on-premise data & services (Azure Application Proxy, Service Bus Relay or Azure App Service BizTalk API Apps Hybrid Connections)
  • Push notifications (Notification Hub)
  • Azure Mobile Services
  • Security & Compliance (Workplace Join, Azure Rights Management, Key Vault)

 

 

Related services can be managed using Azure resource groups defining a lifecycle boundary in a JSON-based template and applying RBAC policies to secure said resources.

Select the appropriate storage option

Read / write patterns often drive complexity in balancing consistency with performance.

  • Immediate consistency or strong consistency uses locks to ensure all observers will see (or not see) updates however this has a demand on process cycles.
  • Eventual consistency allows more immediate access to data however the results may not be current. Read heavy applications may favour this method for performance.
  • Optimistic concurrency allows data writes without locks, the revision number pre-write is stored and when data is written if the revision number has changed the write operation will fail.
  • Pessimistic concurrency is better suited to write-heavy applications as performance is likely to be better as a result of avoiding so many failed write operations.
  • Last-write wins is a simpler method whereby as the name implies the last write operation will be successful.
  • Sequential access efficiently reads data in a continuous range.
  • Random access directly addresses data points by path or a hash code of the data.

Data queries may be simple (using a key to address data) or more complex involving correlations, filters, etc.

  • Static schema relies on an explicit or fixed schema so all parties know how data is structured. A static schema supports complex queries and automation well. When querying large sets of data indexing can help however this can drive down performance.
  • Dynamic schema also known as schema-less or NoSQL databases do not have a fixed schema instead saving data as key-value pairs. A dynamic schema can support greater flexibility by enabling fields to be added, changed and removed without worrying about schema mismatches. However dynamic schemas can struggle with more complex queries.

Repetitive queries can be addressed by using a data cache such as Azure Redis Cache with a Time-To-Live (TTL) specified to quickly return results from memory.

When selecting an appropriate data storage solution consider:

  • Combination of data stores – SQL may be best suited for transactional data, Blob storage for large binary files, DocumentDB for loosely structured data and Azure Search for indexing free-text files
  • Keep data close to compute
  • Cost drivers – performance vs. cost (hot vs. cold, standard vs. premium, etc.)

When evaluating data storage qualities consider:

  • Reliability – LRS (3 local copies), GRS (additional 3 copies in a separate region), Azure SQL uses multiple active secondaries, consider reliability in your own solution
  • Scalability – Data Sharding is a common practice for scaling data stores and providing multi-tenancy. Azure SQL Datbase Elastic Scale supports data sharding.