Thoughts on Integrating Systems & IoT

Posts tagged “Web service

Creating a Web Service in Azure, Part 1- Introduction and Architecture

Introduction

In a series of articles I will describe how Azure can be used to power a back end for a front end application.

My front end application is a simple Rss aggregator.  I follow a number of blogs and I currently use a Windows forms application that I wrote to monitor for new content: the application periodically downloads Rss feeds and processes them on the client side so I can then read them offline (the feed content is stored in a SQLite database).  I would like a better solution that can run on my Android phone and also where feeds can be synced even when the client application isn’t running; the back end will instead be responsible for downloading latest feed content and aggregating the data.

A key design feature is that the new Android client will be lightweight: it will carry out minimal processing of the data and won’t be responsible for downloading feeds from the various blogs.  Such a setup was passable for my high spec laptop but won’t do for my much lower spec phone for these reasons:

  • Downloading feeds from the various blogs will blow out my data plan.
  • Heavy processing on the client side will consume precious CPU cycles and battery power, making my phone slow/unresponsive with a constant need to find the next power outlet to charge it.

So with these limitations in mind, the back end will instead do the “heavy lifting” of downloading and processing feeds and ensure that sync data is optimized to the needs of the client, so minimizing bandwidth consumption

I must also mention as well that while thinking on how Azure could be used to power a back end service, a two part article was published in MSDN magazine that is pretty much along the lines that I was thinking for my own web service (please see the “References” section below for links to these two articles).  The MSDN articles describe a service that aggregates Twitter and StackOverflow data intelligently, while my proof of concept aggregates Rss feed data from blogs, for example.  I draw on these 2 articles heavily in the series.

Another major advantage (mentioned in the MSDN article series) of a cloud back end is better scalability: instead of each client downloading and processing the same feeds individually, the back end application can do this in a single operation, getting around any throttling limitations that may be imposed on some web services.  So as the popularity of an app increases, this doesn’t result in a related decrease in performance (due to throttling) which would damage the reputation of the app.

Architecture

The diagram below shows a high level overview of the solution:

Datamate Architecture

Figure 1  Datamate Architecture (Based on Figure 2 in Reference Article [1])

Some of the key features of the architecture are as follows (walking through the diagram from left to right):

  • Azure SQL Database is used to store Feed data in a relational database and the data is accessed using Entity Framework (EF) via an internal data provider API.  It is envisaged that as further data sources come on board (other than just Rss feeds) each data source (e.g. Twitter) will have it’s own provider API that is implemented to the requirements of the particular data source that is onboarded.
  • Azure WebJobs represent the worker processes – they run as a scheduled background task, downloading and processing Rss feeds and writing the results to the database.
  • A REST API, implemented using ASP.NET Web API, provides an interface for clients to retrieve data.
  • A simple client app (mobile and web) will use the REST API to download data and maintain a client side cache of the data, to the preferences specified by the user, once authenticated and authorised by the REST API.

That’s it for now – stay tuned for part 2!!  In the next post, I will discuss the design and development of the Azure SQL Database and Azure WebJob that represent the “backbone” of the solution.

As always, any comments or tips most welcome.

References

[1] MSDN Magazine Aug 2015 Issue, Create a Web Service with Azure Web Apps and WebJobs, Microsoft.  Available from: https://msdn.microsoft.com/en-us/magazine/mt185572.aspx

[2] MSDN Magazine Sep 2015 Issue, Build a Xamarin App with Authentication and Offline Support, Microsoft.  Available from: https://msdn.microsoft.com/en-us/magazine/mt422581.aspx

Advertisements

BizTalk and WCF, Consuming a WCF Service, Part 2 – The BizTalk WCF Service Consuming Wizard and a Look at the Artefacts Created

BizTalk WCF Service Consuming Wizard - load WSDL for service endpoint

This post builds on part 1 of this series looking at BizTalk and WCF.

The previous post gave an overview of the RandomPresent WCF service, discussing design considerations, hosting and testing using SOAPUI.

In this post, we will examine how to consume the web service using the BizTalk WCF Service Consuming Wizard.  The next post will delve into the detail of actually building our BizTalk solution, using (some) of the artefacts generated by the wizard.

The Scenario

Our mythical company is feeling generous and management have decided that each customer will receive a gift with their order.  The gift will be assigned by calling our newly created web service and then adding an extra order line item for the present.

The BizTalk WCF Service Consuming Wizard

To consume the RandomPresent web service, the BizTalk WCF Service Consuming Wizard could be used.  This will generate all the artefacts we need to invoke the service, into our BizTalk solution.  However, some of the artefacts generated we will discard in favour of our own (better) implementation (to be discussed in part 3).

Here are the steps below:

  1. Right click on your solution (typically an external schemas project) and select Add –> Add Generated Items
  2. Select Consume WCF Service in the Add Generated Items dialogue box.  This will fire up the BizTalk WCF Service Consuming Wizard welcome page – click on the Next button
  3. On the next screen of the wizard we have two choices:
    Image

    Choose MEX endpoint

    • Metadata Exchange (MEX) endpoint – this option enables a service description to be downloaded (WSDL file) by pointing the wizard directly to the running RandomPresent web service.  Note from part 1 of this post, that we exposed a MEX endpoint by configuring the services web.config file to allow metadata to be downloaded via a HTTP GET
    • Metadata Files (WSDL and XSD) – I always think of this as a second best option, since instead of downloading service data directly from source, this option will allow artefacts to consume the service to be created from a WSDL or XSD on the file system.  There is a risk that the WSDL and/or XSD that has been obtained is out of date, so I try and avoid using this function if possible (it’s a last resort :-))
  4. Ensure that the Metadata Exchange (MEX) endpoint option is selected and click Next – this will take you to a screen where the service WSDL can be obtained and loaded into the wizard:
    Load WSDL for service endpoint

    Load WSDL for service endpoint – enter WSDL URL and click “Get” (for HTTP GET)

    • (Note that it is common practice for a service endpoint to expose it’s service definition using the convention servicename?wsdl)
  5. Click Next and then Import.  The wizard will then process the WSDL and generate artefacts into our BizTalk solution.  Click Finish.

The Ajax.BT.Fulfilment BizTalk Solution

So the wizard has consumed our WSDL and generated various BizTalk solution artefacts into our BizTalk purchase order fulfilment solution.  Lets take a quick look at what has been created:

BizTalk solution artefacts created by the BizTalk WCF Service Consuming Wizard

BizTalk solution artefacts created by the BizTalk WCF Service Consuming Wizard

  1. 2 x bindings files for importing into our BizTalk application:  one of these files can be imported into our eventual BizTalk application to create a send port to communicate with the RandomPresent service:
    • RandomPresentService.BindingInfo.xml – from the WSDL, the wizard has detected that the WCF service implements wsHttpBinding and therefore this file will create a send port using the WCF-WSHttp adapter
    • RandomPresentService_Custom.BindingInfo.xml – this is another option for creating a send port in our BizTalk application.  This will create a send port which will use the WCF-Custom adapter.  Utilising this adapter offers greater WCF extensibility compared to the WCF-WSHttp adapter
    • An interesting conundrum is: what bindings should I use in my solution, that utilising WCF-WSHttp or WCF-Custom?  I would contend that WCF-Custom is the better option over WCF-WSHttp, to support future/evolving requirements
  2. RandomPresentService.odx – this is a “starter” orchestration to call the service and contains just the types required.  I always move this orchestration into my specific orchestrations project (or just create an orchestration from scratch)
  3. RandomPresentService_schemas_ajax_randompresentservice.xsd – this schema defines the WCF message types required to construct a request to the service and what we can expect the response to look like
  4. RandomPresentService_schemas_microsoft_com_2003_10_Serialization.xsd – this contains type details.  I suppose if you wish to serialize to a class representation, this XSD would be useful but otherwise I haven’t found a use for this

That’s it for now…  In part 3 of this post I will walk you through my BizTalk solution utilising the service.  This will build on artefacts created by the BizTalk WCF Service Consuming Wizard, which was the main topic of this post.