I have my first application that gets data from a web service, I had managed to get this all working after a lot of pain.
I am a total newb to all this and this is a collabartive development with one of system providers who are php developers. Since doing the ground work our system providers have finished testing there end so all I needed to do was change the requestUri to
the new and I could start retrieving live data.
However when I come to test my app this time I get nothing but errors. Sad times.
I checked the json response in a browser for the url I am querying and everything looks fine, data is return as before. E.g.
using System;
using System.IO;
using System.Net;
using System.Runtime.Serialization.Json;
using Testing_5.Filters;
using Testing_5.Models;
namespace Testing_5.Interface
{
public class JsonHelper
{
public void GetData()
{
ServicePointManager.ServerCertificateValidationCallback = TrustHttps.TrustAllCertificateCallback;
Uri RequestUri = new Uri("https://example/api/completions?token=123456");
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(RequestUri);
request.BeginGetResponse(new AsyncCallback(ResponseHandler), request);
}
private void ResponseHandler(IAsyncResult iar)
{
HttpWebRequest request = (HttpWebRequest)iar.AsyncState;// validate cert by calling a function
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(iar);
using (Stream responseStream = response.GetResponseStream())
{
DataContractJsonSerializer dcjs = new DataContractJsonSerializer(typeof(SkillzResult));
SkillzResult isr = (SkillzResult)dcjs.ReadObject(responseStream);
}
}
}
}
Models:
public class ResultSet
{
public SkillzResultSet ResultSetGrp { get; set; }
}
public class SkillzResultSet
{
public int totalResultsAvailable { get; set; }
public int totalResultsReturned { get; set; }
public int firstResultPosition { get; set; }
public SkillzResult[] Result { get; set; }
}
public class SkillzResult
{
[Key]
public string UserName { get; set; }
[Required]
public string LastName { get; set; }
[Required]
public string FirstName { get; set; }
[Required]
public string Name { get; set; }
[Email]
[Required]
public string Email { get; set; }
[Required]
public string CourseName { get; set; }
[Required]
public DateTime Created { get; set; }
[Min(0, ErrorMessage = "The least you can score is zero.")]
[Max(100, ErrorMessage = "Max score is 100.")]
public int? Score { get; set; }
}
Trust SSL:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;
namespace Testing_5.Filters
{
public class TrustHttps
{
public static bool TrustAllCertificateCallback(object sender, X509Certificate cert, X509Chain chain, SslPolicyErrors errors)
{
return true;
}
}
}
Controller Read action:
[Filters.RequireHttps]
[HttpGet]
public ActionResult Read()
{
Interface.JsonHelper jHelp = new Interface.JsonHelper();
jHelp.GetData();
var skills = unitOfWork.SkillzResultRepository.Get()
// Use a view model to avoid serializing internal Entity Framework properties as JSON
.Select(p => new SkillzResult
{
UserName = p.UserName,
LastName = p.LastName,
FirstName = p.FirstName,
Name = p.Name,
Email = p.Email,
CourseName = p.CourseName,
Created = p.Created,
Score = p.Score
})
.ToList();
return Json(skills, JsonRequestBehavior.AllowGet);
}
The error I get is in the JsonHelper ResponseHandler method, in the using loop after I step through the line begining SkillsResult isr, it skips out of the using loop to the end of the void. This seems like it is not receiving anything but I can't understand
why?
I added a some code in to retrive the CourseName from the responseStream, but it was null?
If you use defaults, the json formatter will use Json.net (uses data contract serializer by default if I remember correctly). Also assuming that the default response content type is json or that you are explicitly setting accept. If these assumptions are
not true, you can naturally force the response to use a specifc formatter for reading the content (overload that accepts formatter).
Please "Mark as Answer" if this resolves your query. Thanks!
Thanks thats really helpful, I don't think I have tried HttpClient yet I am still really new to this.
Does that method handle dynamic json as mine is? e.g.
{ Status, Data [{ User, Name, Result ... }] }
I managed to get it working but I don't know if what I have does is the correct way of doing things (my code is at the bottom if you would be kind enough to review it)?
Another thing in terms of security I am using a function to accept the SSL, is this correct or does this open me up to security issues and if so what is the correct way of dealing with this?
Sorry fall all the questions, my final one is the next stage of my web api is to update user data on their system. The way there system does this is by reading a CSV file. What I have to do is serialise my entire user set from the HR system in to CSV and
then POST the CSV to their RESTful web api. My question is, can I searialise a query in to CSV on the fly and how do I then POST it to a web api? I don't really want to save the CSV file as I have no need for this data so if it could be done virtually that
would be great! I have been looking at custom media formatters but really can't get my head around them and so far all my attempts have failed.
Thanks again for the help,
Andy
Read Action:
[Filters.RequireHttps]
[HttpPost]
public ActionResult Read()
{
ServicePointManager.ServerCertificateValidationCallback = TrustHttps.TrustAllCertificateCallback;
HttpResponseMessage response = client.GetAsync("https://example/api/completions?token=123456").Result;
response.EnsureSuccessStatusCode();
SkillzStatus skillzStatus = JsonConvert.DeserializeObject<SkillzStatus>(response.Content.ReadAsStringAsync().Result);
var result = new List<SkillzResult>();
//Iterate all created products which are posted by the Kendo Grid
foreach (var skillzresult in skillzStatus.data)
{
if (unitOfWork.SkillzResultRepository.Get(m => m.username == skillzresult.username).Select(c => c.course_name == skillzresult.course_name && c.created == skillzresult.created).Count() == 0)
{
try
{
// Create a new Product entity and set its properties from productViewModel
var skill = new SkillzResult
{
username = skillzresult.username,
last_name = skillzresult.last_name,
first_name = skillzresult.first_name,
name = skillzresult.name,
email = skillzresult.email,
course_name = skillzresult.course_name,
created = skillzresult.created,
result = skillzresult.result
};
// store the product in the result
result.Add(skill);
// Add the entity
unitOfWork.SkillzResultRepository.Insert(skill);
}
catch (Exception e)
{
Console.WriteLine(e.Data + " - " + e.Message);
}
}
}
Require Https:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.Http.Filters;
using System.Web.Http.Controllers;
using System.Text;
using System.Net.Http;
using System.Net;
namespace Testing_5.Filters
{
public class RequireHttpsAttribute : AuthorizationFilterAttribute
{
public override void OnAuthorization(HttpActionContext actionContext)
{
var request = actionContext.Request;
if (request.RequestUri.Scheme != Uri.UriSchemeHttps)
{
HttpResponseMessage response;
UriBuilder uri = new UriBuilder(request.RequestUri);
uri.Scheme = Uri.UriSchemeHttps;
uri.Port = 443;
string body = string.Format("<p>The resource can be found at <a href=\"{0}\">{0}</a>.</p>",
uri.Uri.AbsoluteUri);
if (request.Method.Equals(HttpMethod.Get) || request.Method.Equals(HttpMethod.Head))
{
response = request.CreateResponse(HttpStatusCode.Found);
response.Headers.Location = uri.Uri;
if (request.Method.Equals(HttpMethod.Get))
{
response.Content = new StringContent(body, Encoding.UTF8, "text/html");
}
}
else
{
response = request.CreateResponse(HttpStatusCode.NotFound);
response.Content = new StringContent(body, Encoding.UTF8, "text/html");
}
actionContext.Response = response;
}
}
}
}
Trust Https:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;
namespace Testing_5.Filters
{
public class TrustHttps
{
public static bool TrustAllCertificateCallback(object sender, X509Certificate cert, X509Chain chain, SslPolicyErrors errors)
{
return true;
}
}
}
Why don't you abstract the call to the second service via a separate class (e.g. CompletionsProxy or something)? The will decouple the logic, and also support IOC etc. if you like.
Yes, it's dangerous to trust all the SSL certs - only advised to do so in the test environment.
When you do client.GetAsync.Result directly, it's a blocking call and if you're reusing the HttpClient instance across the domain, you'll face issues. Alternative is client.GetAsync.ContinueWith...
response.EnsureSuccessStatusCode() will throw an exception if status code != 200. If you want a more graceful way of handling this, check respone.StatusCode and take action accordingly.
SkillzStatus skillzStatus = JsonConvert.DeserializeObject<SkillzStatus>(response.Content.ReadAsStringAsync().Result); - Instead pof doing this, you could also use
response.ReadContent( typeof(SkillzStatus),new List<MediaTypeFormatter>(GlobalConfiguration.Configuration.Formatters.JsonFormatter)). In fact, if response content type header is already set to Json, you can simply use the ReadContent<T>() method.
As for CSV, you would need a custom formatter supporting a media type like say “text\csv” – it should know how to convert object to csv and vice-versa. Make sure you register it in the formatters collection. E.g. http://www.tugberkugurlu.com/archive/creating-custom-csvmediatypeformatter-in-asp-net-web-api-for-comma-separated-values-csv-format
Hope this helps.
Cheers,
Priya
Please "Mark as Answer" if this resolves your query, Thanks!
Please "Mark as Answer" if this resolves your query. Thanks!
Marked as answer by mcinnes01 on Dec 13, 2012 11:45 AM
I will give it a go and see how I get on, I will definately abstract this as you recommend.
In terms of the SSL how do I get it to allow it for this site? Do I need to install the certificate somewhere? Can this be done in the project or do I need to do it on both my development machine whilst test and the live server once I deploy?
I will try the GetAsync.ContinueWith and let you know how I get on.
Definately agree with handling the exceptions and I have noticed this a few time such as 503 when they have stopped the service.
I have read tugberkugurlu's post a few times but my attempts have not been sucessful so far, perhaps after I have done the above I will have a better understanding of this works and might have a little more sucess with the custom media formatter.
For the SSL - is this your own test service or a 3rd party? If it's using a real certificate, you shouldn't have an issue since it will probably have been issued by a trusted authority. For test purposes, you can continue to use what you're currently using.
Maybe you could use conditional compile time constants to determine whether or not you should accept all SSL certs (so you return True in test/debug mode)?
Alternatively, you need to place the cert in your certificate store - and if its self -signed, I think you also need to add to Trusted Root CA store. Please don't do this if you don't trust the issuer etc. though.
Please "Mark as Answer" if this resolves your query. Thanks!
mcinnes01
Member
16 Points
121 Posts
HttpResponse Stream Issues
Dec 12, 2012 02:52 PM|LINK
Hi,
I have my first application that gets data from a web service, I had managed to get this all working after a lot of pain.
I am a total newb to all this and this is a collabartive development with one of system providers who are php developers. Since doing the ground work our system providers have finished testing there end so all I needed to do was change the requestUri to the new and I could start retrieving live data.
However when I come to test my app this time I get nothing but errors. Sad times.
I checked the json response in a browser for the url I am querying and everything looks fine, data is return as before. E.g.
{"status":"OK", "data":[{"username":123456, "last_name":"Smith", "first_name":"Alan", "name":"SASAD", "email":"ASmith@example.com", "course_name":"Fire Safety Essentials", "created":"06-12-2012", "result":100}, {"username":123456, "last_name":"Smith", "first_name":"Alan", "name":"SASAD", "email":"ASmith@example.com", "course_name":"Health and Safety Induction ", "created":"07-12-2012", "result":100}, {"username":654321, "last_name":"Bloggs", "first_name":"Joe", "name":"SASAD", "email":"JBloggs@example.com", "course_name":"Fire Safety Essentials", "created":"11-12-2012", "result":100}, {"username":654321, "last_name":"Bloggs", "first_name":"Joe", "name":"SASAD", "email":"JBloggs@example.com", "course_name":"Health and Safety Induction ", "created":"11-12-2012", "result":100}] }
This is my code:
using System; using System.IO; using System.Net; using System.Runtime.Serialization.Json; using Testing_5.Filters; using Testing_5.Models; namespace Testing_5.Interface { public class JsonHelper { public void GetData() { ServicePointManager.ServerCertificateValidationCallback = TrustHttps.TrustAllCertificateCallback; Uri RequestUri = new Uri("https://example/api/completions?token=123456"); HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(RequestUri); request.BeginGetResponse(new AsyncCallback(ResponseHandler), request); } private void ResponseHandler(IAsyncResult iar) { HttpWebRequest request = (HttpWebRequest)iar.AsyncState;// validate cert by calling a function HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(iar); using (Stream responseStream = response.GetResponseStream()) { DataContractJsonSerializer dcjs = new DataContractJsonSerializer(typeof(SkillzResult)); SkillzResult isr = (SkillzResult)dcjs.ReadObject(responseStream); } } } }Models:
public class ResultSet { public SkillzResultSet ResultSetGrp { get; set; } } public class SkillzResultSet { public int totalResultsAvailable { get; set; } public int totalResultsReturned { get; set; } public int firstResultPosition { get; set; } public SkillzResult[] Result { get; set; } } public class SkillzResult { [Key] public string UserName { get; set; } [Required] public string LastName { get; set; } [Required] public string FirstName { get; set; } [Required] public string Name { get; set; } [Email] [Required] public string Email { get; set; } [Required] public string CourseName { get; set; } [Required] public DateTime Created { get; set; } [Min(0, ErrorMessage = "The least you can score is zero.")] [Max(100, ErrorMessage = "Max score is 100.")] public int? Score { get; set; } }Trust SSL:
using System; using System.Collections.Generic; using System.Linq; using System.Web; using System.Net.Security; using System.Security.Cryptography.X509Certificates; namespace Testing_5.Filters { public class TrustHttps { public static bool TrustAllCertificateCallback(object sender, X509Certificate cert, X509Chain chain, SslPolicyErrors errors) { return true; } } }Controller Read action:
[Filters.RequireHttps] [HttpGet] public ActionResult Read() { Interface.JsonHelper jHelp = new Interface.JsonHelper(); jHelp.GetData(); var skills = unitOfWork.SkillzResultRepository.Get() // Use a view model to avoid serializing internal Entity Framework properties as JSON .Select(p => new SkillzResult { UserName = p.UserName, LastName = p.LastName, FirstName = p.FirstName, Name = p.Name, Email = p.Email, CourseName = p.CourseName, Created = p.Created, Score = p.Score }) .ToList(); return Json(skills, JsonRequestBehavior.AllowGet); }The error I get is in the JsonHelper ResponseHandler method, in the using loop after I step through the line begining SkillsResult isr, it skips out of the using loop to the end of the void. This seems like it is not receiving anything but I can't understand why?
I added a some code in to retrive the CourseName from the responseStream, but it was null?
I would really apprciate your help,
Andy
priya_marwah...
Member
81 Points
52 Posts
Re: HttpResponse Stream Issues
Dec 13, 2012 07:14 AM|LINK
Hi Andy,
Instead of using HttpWebRequest, have you tried using HttpClient? Something like this:
client.GetAsync(uri).ContinueWith(t=> {
if (t.IsCompleted)
{
var response = t.Result; // this will give you the response stream
try
{
skillResults= response.ReadContent<SkillzResult>();
}
catch (Exception ex)
{
// do something
}
}
});
If you use defaults, the json formatter will use Json.net (uses data contract serializer by default if I remember correctly). Also assuming that the default response content type is json or that you are explicitly setting accept. If these assumptions are not true, you can naturally force the response to use a specifc formatter for reading the content (overload that accepts formatter).
mcinnes01
Member
16 Points
121 Posts
Re: HttpResponse Stream Issues
Dec 13, 2012 09:37 AM|LINK
Hi Priya,
Thanks thats really helpful, I don't think I have tried HttpClient yet I am still really new to this.
Does that method handle dynamic json as mine is? e.g.
{ Status, Data [{ User, Name, Result ... }] }
I managed to get it working but I don't know if what I have does is the correct way of doing things (my code is at the bottom if you would be kind enough to review it)?
Another thing in terms of security I am using a function to accept the SSL, is this correct or does this open me up to security issues and if so what is the correct way of dealing with this?
Sorry fall all the questions, my final one is the next stage of my web api is to update user data on their system. The way there system does this is by reading a CSV file. What I have to do is serialise my entire user set from the HR system in to CSV and then POST the CSV to their RESTful web api. My question is, can I searialise a query in to CSV on the fly and how do I then POST it to a web api? I don't really want to save the CSV file as I have no need for this data so if it could be done virtually that would be great! I have been looking at custom media formatters but really can't get my head around them and so far all my attempts have failed.
Thanks again for the help,
Andy
Read Action:
[Filters.RequireHttps] [HttpPost] public ActionResult Read() { ServicePointManager.ServerCertificateValidationCallback = TrustHttps.TrustAllCertificateCallback; HttpResponseMessage response = client.GetAsync("https://example/api/completions?token=123456").Result; response.EnsureSuccessStatusCode(); SkillzStatus skillzStatus = JsonConvert.DeserializeObject<SkillzStatus>(response.Content.ReadAsStringAsync().Result); var result = new List<SkillzResult>(); //Iterate all created products which are posted by the Kendo Grid foreach (var skillzresult in skillzStatus.data) { if (unitOfWork.SkillzResultRepository.Get(m => m.username == skillzresult.username).Select(c => c.course_name == skillzresult.course_name && c.created == skillzresult.created).Count() == 0) { try { // Create a new Product entity and set its properties from productViewModel var skill = new SkillzResult { username = skillzresult.username, last_name = skillzresult.last_name, first_name = skillzresult.first_name, name = skillzresult.name, email = skillzresult.email, course_name = skillzresult.course_name, created = skillzresult.created, result = skillzresult.result }; // store the product in the result result.Add(skill); // Add the entity unitOfWork.SkillzResultRepository.Insert(skill); } catch (Exception e) { Console.WriteLine(e.Data + " - " + e.Message); } } }Require Https:
using System; using System.Collections.Generic; using System.Linq; using System.Web; using System.Web.Http.Filters; using System.Web.Http.Controllers; using System.Text; using System.Net.Http; using System.Net; namespace Testing_5.Filters { public class RequireHttpsAttribute : AuthorizationFilterAttribute { public override void OnAuthorization(HttpActionContext actionContext) { var request = actionContext.Request; if (request.RequestUri.Scheme != Uri.UriSchemeHttps) { HttpResponseMessage response; UriBuilder uri = new UriBuilder(request.RequestUri); uri.Scheme = Uri.UriSchemeHttps; uri.Port = 443; string body = string.Format("<p>The resource can be found at <a href=\"{0}\">{0}</a>.</p>", uri.Uri.AbsoluteUri); if (request.Method.Equals(HttpMethod.Get) || request.Method.Equals(HttpMethod.Head)) { response = request.CreateResponse(HttpStatusCode.Found); response.Headers.Location = uri.Uri; if (request.Method.Equals(HttpMethod.Get)) { response.Content = new StringContent(body, Encoding.UTF8, "text/html"); } } else { response = request.CreateResponse(HttpStatusCode.NotFound); response.Content = new StringContent(body, Encoding.UTF8, "text/html"); } actionContext.Response = response; } } } }Trust Https:
using System; using System.Collections.Generic; using System.Linq; using System.Web; using System.Net.Security; using System.Security.Cryptography.X509Certificates; namespace Testing_5.Filters { public class TrustHttps { public static bool TrustAllCertificateCallback(object sender, X509Certificate cert, X509Chain chain, SslPolicyErrors errors) { return true; } } }priya_marwah...
Member
81 Points
52 Posts
Re: HttpResponse Stream Issues
Dec 13, 2012 11:09 AM|LINK
Hi Andy,
response.ReadContent( typeof(SkillzStatus),new List<MediaTypeFormatter>(GlobalConfiguration.Configuration.Formatters.JsonFormatter)). In fact, if response content type header is already set to Json, you can simply use the ReadContent<T>() method.
Hope this helps.
Cheers,
Priya
Please "Mark as Answer" if this resolves your query, Thanks!
mcinnes01
Member
16 Points
121 Posts
Re: HttpResponse Stream Issues
Dec 13, 2012 11:53 AM|LINK
Thanks Priya,
I will give it a go and see how I get on, I will definately abstract this as you recommend.
In terms of the SSL how do I get it to allow it for this site? Do I need to install the certificate somewhere? Can this be done in the project or do I need to do it on both my development machine whilst test and the live server once I deploy?
I will try the GetAsync.ContinueWith and let you know how I get on.
Definately agree with handling the exceptions and I have noticed this a few time such as 503 when they have stopped the service.
I have read tugberkugurlu's post a few times but my attempts have not been sucessful so far, perhaps after I have done the above I will have a better understanding of this works and might have a little more sucess with the custom media formatter.
Thanks again,
Andy
priya_marwah...
Member
81 Points
52 Posts
Re: HttpResponse Stream Issues
Dec 13, 2012 12:10 PM|LINK
For the SSL - is this your own test service or a 3rd party? If it's using a real certificate, you shouldn't have an issue since it will probably have been issued by a trusted authority. For test purposes, you can continue to use what you're currently using. Maybe you could use conditional compile time constants to determine whether or not you should accept all SSL certs (so you return True in test/debug mode)?
Alternatively, you need to place the cert in your certificate store - and if its self -signed, I think you also need to add to Trusted Root CA store. Please don't do this if you don't trust the issuer etc. though.