In this post, I will be showing the progress of Social KPI Application so far through a screencast. This is my first screencast, so learning about OBS (Open Broadcast Software) along the way.
Lately, I have been working on a few interesting problems that I wanted to talk about. So I started studying more of JAXB APIs and the whole rabbit hole is huge. There are a lot of things that I can focus on these APIs. In this post, I will show how to use JAXB APIs for XML processing.
First, let me talk about JAXB APIs. So XML has been a language for data exchange for some time now. Java and XML came up with a partner to help developers a standard to facilitate between applications, web services, organizations using a similar technology stack. So all developers came together and they built a standard API for Java Architecture for XML Binding (JAXB). I will not describe any other detail about these API in this post, but you can check out this official documentation JAXB. JAXB APIs help to convert XML documents into Java Object and vice versa.
In this scenario, how do bind a schema to Java object?
JAXB basically simplifies the access of the XML document in Java program. So as part of this, we bind schema for XML document into a set of Java classes.
To demonstrate this example, I am showing a sample XML books.xml as below:
<bookstore>
<book category="cooking">
<title lang="en">Everyday Italian</title>
<author>Giada De Laurentiis</author>
<year>2005</year>
<price>30.00</price>
</book>
<book category="children">
<title lang="en">Harry Potter</title>
<author>J K. Rowling</author>
<year>2005</year>
<price>29.99</price>
</book>
<book category="web">
<title lang="en">XQuery Kick Start</title>
<author>James McGovern</author>
<author>Per Bothner</author>
<author>Kurt Cagle</author>
<author>James Linn</author>
<author>Vaidyanathan Nagarajan</author>
<year>2003</year>
<price>49.99</price>
</book>
<book category="web" cover="paperback">
<title lang="en">Learning XML</title>
<author>Erik T. Ray</author>
<year>2003</year>
<price>39.95</price>
</book>
</bookstore>
To convert this XML into Java object, we will follow the following process
JAXBContext ObjectUnmarshaller using JAXBContext ObjectUnmarshal method on Unmarshaller
package com.betterjavacode.jaxbexample;
import java.util.ArrayList;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlElementWrapper;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlType;
@XmlRootElement
@XmlType(propOrder = {"bookcategory"})
public class Bookstore
{
private String bookCategory;
private List books;
public Bookstore(){}
public String getBookCategory()
{
return this.bookCategory
}
public void setBookCategory(String bookCategory)
{
this.bookCategory = bookCategory;
}
public List getBooks()
{
return this.books;
}
public void setBooks(List books)
{
this.books = books;
}
}
package com.betterjavacode.jaxbexample;
import javax.xml.bind.annotation.XmlRootElement;
@XmlRootElement(namespace = "com.betterjavacode.jaxbexample.Bookstore")
public class Book
{
private String title;
private String author;
private int year;
private long price;
public Book()
{
}
public Book(String title, String author, int year, long price)
{
this.title = title;
this.author = author;
this.year = year;
this.price = price;
}
public String getTitle()
{
return title;
}
public void setTitle(String title)
{
this.title = title;
}
public String getAuthor()
{
return author;
}
public void setAuthor(String author)
{
this.author = author;
}
public int getYear()
{
return year;
}
public void setYear(int year)
{
this.year = year;
}
public long getPrice()
{
return price;
}
public void setPrice(long price)
{
this.price = price;
}
}
Using annotations and base classes, we can convert xml into java object easily. That process as described will look like below in program:
package com.betterjavacode.jaxbexample;
import java.io.File;
import java.util.ArrayList;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Unmarshaller;
public class JAXBXMLToJava
{
public static final Logger LOGGER = LogFactory.getLogger(JAXBXMLToJava.class);
public static void main(String[] args)
{
try
{
JAXBContext jaxbContext = JAXBContext.newInstance(Bookstore.class);
Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();
File xmlFile = new File("C:\\temp\\books.xml");
Bookstore bookStore = (Bookstore) unmarshaller.unmarshal(xmlFile);
ArrayList books = bookStore.getBooks();
for(Book book:books)
{
System.out.println("Book: " + book.getTitle());
}
}
catch(JAXBException e)
{
LOGGER.debug("unable to unmarshal", e);
}
}
}
So in this scenario, we took an xml and converted that into a java object.
In this scenario, I will show one of the features of JAXB APIs.
How to change the namespace prefixes?
By default when you marshal Java object to XML, JAXB Marshaller uses default namespace prefixes like ns1, ns2, ns3 etc. While this can be perfectly valid XML for readability purposes, you might run into an issue like signature validation if Signature validation API was expecting different namespaces.
In such cases, JAXB offers a NamespacePrefixMapper which allows you to change your namespace prefixes.
Previously, I had shown how to consume a SOAP webservice using JAXB marshaller. In this post, I have given a brief introduction to JAXB APIs and how to use them for XML processing. There are a lot of other features that JAXB APIs offer.
Lately, I have been hearing a lot about Kafka streaming. Even though I have worked on microservices, I haven’t really tackled heavy data applications. In my previous experience, where we did deal with heavy data for health insurance benefits, it was very different.
Now with Netflix and Amazon, data streaming has become a major target. With growing technology and information, it has become even more important to tackle the growing data. In simple terms, all web applications should be able to process large data sets with improved performance. Data sets size should not deter applications usage.
Firstly we used to process large data in batch, but that is not continuous processing and sometimes, it doesn’t work in real-time scenarios for applications. Like Netflix, batch processing will never work. What’s the alternative? The data streaming. Data streaming is a process of sending data sets continuously. This process is the backbone for applications like Netflix and Amazon. Also the growing social network platforms, data streaming is at the heart of handling large data.
The streamed data is often used for real-time aggregation and correlation, filtering, or sampling. One of the major benefits of data streaming is that it allows us to view and analyze data in real-time.
Few challenges that data streaming often faces
There are a bunch of tools that are available for data streaming. Amazon offers Kinesis, Apache has few open-source tools like Kafka, Storm, and Flink. Similarly, in future posts, I will talk more about Apache Kafka and its usage. Here I am just giving a brief idea about Apache Kafka.
Apache Kafka is a real-time distributed streaming platform. Basically it allows us to publish and subscribe to streams of records.
There are two main usages where Apache Kafka is used:
Above all, the basic idea that Kafka has adapted is from Hadoop. It runs as a cluster of one or more servers that can span multiple data centers. These clusters store data streams. Each record in the stream comprised of key, value, and timestamp. Kafka provides four main APIs Producer, Consumer, Streams, Connector.
Similarly, in future posts, I will break down these APIs in detail with their usage in a sample application.
Any enterprise application has to perform well under load. So performance of such application can be tracked. There are tools that help to track this performance. Apache JMeter is one such tool. In this post, we will use Apache JMeter to do a load test on a sample application and see how we can measure the performance.
Depending on your application, you can track different functionality for load tests. Database heavy operations would be the most affected by this tracking. That’s where load tests can help us to point out flaws in our design and code and how we can improve. One major difference to note is that load tests are different from stress tests. In stress tests, we try to measure how much load the application can tolerate before it breaks. In load tests, we are measuring the performance from a time perspective.
Start Apache JMeter and record the test script. Let’s create a test plan that we will execute for our load test. As part of this post, I will be load testing login to a Spring boot based web application that I am running at https://localhost:8443/home
Once you launch JMeter, on left hand side window you will see an option for test plan. Create a test plan and let’s name it Test1 as shown below:
Test Plan for JMeter Script
Once you create the test plan, select how many threads (number of users) you want to run for this test, what’s the ramp-up period for one thread and what’s the loop count. For this post, I have selected 50 threads with each thread ramping up in 2 seconds with a single loop.
Right click on test plan and add config elements for HTTP Cache Manager and HTTP Cookie Manager. This is required for handling CSRF tokens in our request that we will need later.
Now to test our web application, we will add a sampler with HTTP Request. We will name this HomePage. This is displayed as below:
Http Request Sampler
We will have to provide protocol, server name, port number and path as shown above.
Now we will add a listener to our sampler to view results as a tree.
We will need _csrf token from our home page so we can pass this token in POST request to login page of web application.
This Post Processor will have a regular expression extractor which will extract _csrf token from home page and will pass to next HTTP Request Sampler for Login Page.
Regular Expression Extractor
Now we will add Sampler for Login Page which will be a POST request to login endpoint of our web application. We will also pass username and password along with _csrf token whose value will be populated from extractor.
Login Page
Now if we run our script in JMeter, it will run 50 threads at the same time to test login for application. Depending on performance, it might take a long time, but considering this a simple application, JMeter will run this test within few seconds.
Once the test is complete, you can check the View Results Tree and it will show you the results of our each individual test thread. In Response Data tab, you can view if our test was successful or not. It should show the result of our page in html format.
Results
In this post, we used Apache JMeter for load testing and how it can help us to design our application from performance perspective.
In this post, I will explain how we can use clockskew in SAML Assertion to avoid a general problem when a Service Provider (SP) receives a SAML response from Identity Provider (IdP) and if both of them are on different machines which is generally the scenario.
So when a user accesses a SP application, SP sends an authentication request to IdP and IdP sends a SAML response to SP. The typical SAML response looks like below:
<saml2p:Response Destination="https://localhost:8443/spring-security-saml2-sample/saml/SSO"
ID="_8ced26c57648ea420c5e27ac7f3d78b3"
InResponseTo="a3ei0bej41ie0dg738jji60271f1bca"
IssueInstant="2018-10-31T18:39:05.109Z"
Version="2.0"
xmlns:saml2p="urn:oasis:names:tc:SAML:2.0:protocol"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
>
<saml2:Issuer xmlns:saml2="urn:oasis:names:tc:SAML:2.0:assertion">https://sts.betterjavacode.com/sts</saml2:Issuer>
<ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
<ds:SignedInfo>
<ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
<ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" />
<ds:Reference URI="#_8ced26c57648ea420c5e27ac7f3d78b3">
<ds:Transforms>
<ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" />
<ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
<ec:InclusiveNamespaces PrefixList="xsd"
xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#"
/>
</ds:Transform>
</ds:Transforms>
<ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256" />
<ds:DigestValue>k6k9seVwZT8qEfainB+HDUJmv7wLwPjJxRARgrFLk3E=</ds:DigestValue>
</ds:Reference>
</ds:SignedInfo>
<ds:SignatureValue>
eYl7/5Bg4f2ZtXYl1Tp/ZW4CIqna1sjHjVSB/zhj3NHncQu7pq9PBTdWz0JEaE8+efICz7trDQvI
J0tih35Vg7NqDtOeps7vMttV0XvrnWKIXEqrscyNgkhwy+KW2oprKIWTJq6y3Z0kQ7n2DlTRkfAE
yLRPxqr8AZU+77Tbv4DmTQVp934ivibUaMNU79nkhMMo7vf0ldpeNCe5Ll5Q7nxgNBCrL4mhbGdU
DNJDVqJIhQZDJUYhBVZSgLo6mYLxf0ndQr5+GdcvO3i8VlooH49I5ZO6LfsBlNiSU6WezC0Fcz8J
biodrT3h62Jny8FKUgYfXA8i0ZoAerCxHmLGFQ==
</ds:SignatureValue>
<ds:KeyInfo>
<ds:X509Data>
<ds:X509Certificate>MIICpTCCAY2gAwIBAgIHAKQk/Tty7TANBgkqhkiG9w0BAQsFADATMREwDwYDVQQDDAhJbmZvclNU
UzAeFw0xNzExMTAxNjIwMjVaFw0xOTExMTAxNjIwMjZaMBMxETAPBgNVBAMMCEluZm9yU1RTMIIB
IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAuy40GofgfSldiu7AXRSWQZo8YIUmdJOXV5gd
YUiIBggcm5J+jttd21av9AWBDLtvekIqgG+nM1SEarDNlCgrnThqDtdsBDnT+B5FQowLwqNQZb0C
6PSWccp484OLHhv3YwbjV/IxgW6wlv2EejPF93NTPc5TkkpIKIEARJwKiUvuwzkcX1atD8HESj8/
5wZhK3g3MSv+CaJb1Y732U2aa34RI6HYRNlKRsqRj55SZnPs3AmIBL/rdVbt5eYJdK+jXSOMQ9Sb
TNkgTAYKnlSQP8NhsNc1Y5ZJ231KD3rSqHEXwpKVXFuEgtBnCL9utqyUU0Fu9PNJJj0hf9otMJw2
pQIDAQABMA0GCSqGSIb3DQEBCwUAA4IBAQAsqxA/+llqNtJq/VPtAa0XwtldB7/4Gew5LgfU5Yj7
/ruHuxCh03xfnLyjo0bxrFeMbhHMg4MzJhgah0zTIl4WMRI781CiMxGfMdvQAHhY5hgpryqKUTbH
qcIoW1WkTEL+TrOSk2gjL/n/4KsUJ3XKeI4j4h2RDvbIF8u9cbrB5RupEwJlo/pK3mCr3GQGFxVX
4yli+AysMmz8JWz2hWQad0QQr9KIYmZdgauNt7uYn4u+7cngtTtQF/EMoZU1pAFI5lDP13mxI1rR
eYGzoKhGfHIy7TiXYvqX0vFomzkgr6D0DnRlRkQR8J5EVtnGZ7sF50KlNviG+IxrPNndUdeX</ds:X509Certificate>
</ds:X509Data>
</ds:KeyInfo>
</ds:Signature>
<saml2p:Status>
<saml2p:StatusCode Value="urn:oasis:names:tc:SAML:2.0:status:Success" />
</saml2p:Status>
<saml2:Assertion ID="_43514fee402111bfa8ef07e2bb3e1816"
IssueInstant="2018-10-31T18:39:05.109Z"
Version="2.0"
xmlns:saml2="urn:oasis:names:tc:SAML:2.0:assertion"
>
<saml2:Issuer>https://sts.betterjavacode.com/sts</saml2:Issuer>
<saml2:Subject>
<saml2:NameID Format="urn:oasis:names:tc:SAML:2.0:nameid-format:transient"
NameQualifier="https://sts.betterjavacode.com/sts"
SPNameQualifier="https://localhost:8443/spring-security-saml2-sample/saml/metadata"
>-74375301938178262591541011144180</saml2:NameID>
<saml2:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer">
<saml2:SubjectConfirmationData InResponseTo="a3ei0bej41ie0dg738jji60271f1bca"
NotOnOrAfter="2018-10-31T18:44:05.109Z"
Recipient="https://localhost:8443/spring-security-saml2-sample/saml/SSO"
/>
</saml2:SubjectConfirmation>
</saml2:Subject>
<saml2:Conditions NotBefore="2018-10-31T18:39:05.109Z"
NotOnOrAfter="2018-10-31T18:44:04.109Z"
>
<saml2:AudienceRestriction>
<saml2:Audience>https://localhost:8443/spring-security-saml2-sample/saml/metadata</saml2:Audience>
</saml2:AudienceRestriction>
</saml2:Conditions>
<saml2:AttributeStatement>
<saml2:Attribute Name="http://schemas.microsoft.com/claims/Tenant">
<saml2:AttributeValue xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:type="xsd:string"
>ABC Company</saml2:AttributeValue>
</saml2:Attribute>
<saml2:Attribute Name="http://schemas.microsoft.com/claims/Identity">
<saml2:AttributeValue xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:type="xsd:string"
>testuser</saml2:AttributeValue>
</saml2:Attribute>
<saml2:Attribute Name="http://schemas.xmlsoap.org/claims/CommonName">
<saml2:AttributeValue xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:type="xsd:string"
>Test User</saml2:AttributeValue>
</saml2:Attribute>
<saml2:Attribute Name="http://schemas.microsoft.com/claims/Email">
<saml2:AttributeValue xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:type="xsd:string"
>test.user@betterjavacode.com</saml2:AttributeValue>
</saml2:Attribute>
<saml2:Attribute Name="http://schemas.microsoft.com/claims/Session">
<saml2:AttributeValue xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:type="xsd:string"
>-65311121040491585821541011144177</saml2:AttributeValue>
</saml2:Attribute>
</saml2:AttributeStatement>
<saml2:AuthnStatement AuthnInstant="2018-10-31T18:39:05.109Z"
SessionIndex="31079178551135950171541011144177"
>
<saml2:AuthnContext>
<saml2:AuthnContextClassRef>urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport</saml2:AuthnContextClassRef>
</saml2:AuthnContext>
</saml2:AuthnStatement>
</saml2:Assertion>
</saml2p:Response>
Now SP and IdP are on different machine. And there can be a case that their clocks are not synced. So when IdP generates the response, it has an element SubjectConfirmationData that contains NotBefore and NotOnOrAfter. Subject indicates details about the user and relying party who is receiving the assertion. Relying party has to verify that the assertion is coming from the right issuer.
NotBefore – Time before which subject can not be validated.
NotOnOrAfter – Time on or after which subject has expired.
Similarly, there will be Conditions element in the assertion and this element will also contain attributes NotBefore or NotOnOrAfter. These two attributes indicate at what time assertion will be valid and after what time, it will be expired.
Now if we take into account a scenario where system clocks of IdP and SP are not in sync and if Idp clock is ahead of Sp clock, how do we handle this scenario?
So as described in above scenario, then assertion validation will fail because of NotBefore time will be greater than the time on SP machine.
To solve this issue, we can either make sure IdP and SP clocks are in sync. But this seems human-dependent solution and getting clocks in sync is not always practical solution.
Another way this can be solved by adding a clockskew in SAML assertion. A relying party that is consuming the assertion should apply small clockskew to time to accommodate small changes in time.
private int clockSkew = 120;
SubconfirmationData subjectConfirmationData = subjectConfirmation.getSubjectConfirmationData();
if(subjectConfirmationData.getNotOnOrAfter().plusSeconds(clockSkew).isBeforeNow())
{
// throw error - Subject Confirmation Data has expired
}
/***
*
*
*/
if (conditions.getNotBefore() != null && conditions.getNotBefore().minusSeconds(clockSkew).isAfterNow())
{
throw new SAMLException("Assertion condition notBefore is not valid");
}
if (conditions.getNotOnOrAfter() != null && conditions.getNotOnOrAfter().plusSeconds(clockSkew).isBeforeNow())
{
throw new SAMLException("Assertion condition notOnOrAfter is not valid");
}