Author Archives: yogesh.mali@gmail.com

All about Kafka Streaming

Lately, I have been hearing a lot about Kafka streaming. Even though I have worked on microservices, I haven’t really tackled heavy data applications. In my previous experience, where we did deal with heavy data for health insurance benefits, it was very different.

Now with Netflix and Amazon, data streaming has become a major target. With growing technology and information, it has become even more important to tackle the growing data. In simple terms, all web applications should be able to process large data sets with improved performance. Data sets size should not deter applications usage.

What is Kafka Data Streaming?

Firstly we used to process large data in batch, but that is not continuous processing and sometimes, it doesn’t work in real-time scenarios for applications. Like Netflix, batch processing will never work. What’s the alternative? The data streaming. Data streaming is a process of sending data sets continuously. This process is the backbone for applications like Netflix and Amazon. Also the growing social network platforms, data streaming is at the heart of handling large data.

The streamed data is often used for real-time aggregation and correlation, filtering, or sampling. One of the major benefits of data streaming is that it allows us to view and analyze data in real-time.

Few challenges that data streaming often faces

  1. Scalability
  2. Durability of data
  3. Fault Tolerance

Tools for Data Streaming

There are a bunch of tools that are available for data streaming. Amazon offers Kinesis, Apache has few open-source tools like Kafka, Storm, and Flink. Similarly, in future posts, I will talk more about Apache Kafka and its usage. Here I am just giving a brief idea about Apache Kafka.

Apache Kafka Streaming

Apache Kafka is a real-time distributed streaming platform. Basically it allows us to publish and subscribe to streams of records.

There are two main usages where Apache Kafka is used:

  1. Building data pipelines where records are streamed continuously.
  2. Building applications that can consume data pipelines and react accordingly

Above all, the basic idea that Kafka has adapted is from Hadoop. It runs as a cluster of one or more servers that can span multiple data centers. These clusters store data streams. Each record in the stream comprised of key, value, and timestamp. Kafka provides four main APIs Producer, Consumer, Streams, Connector.

Similarly, in future posts, I will break down these APIs in detail with their usage in a sample application.

References

  1. Apache Kafka – Kafka
  2. Data Streaming – Data Streaming
  3. Stream Processing – Stream Processing

 

 

 

How to send email with Spring Boot

Scenario

In this post, I show how to use email configuration inside a spring boot application. The use case I want to discuss here is if you have Contact Us page on your web application where you allow users to send email to your sales team or support team, then how do you accomplish this feature about sending an email with contact us form using Spring Boot.

What will you need

  • Java 8
  • IntelliJ
  • A Spring boot based web application

Use case solution

As part of this post, I will not be describing how to build a spring boot based web application. You can visit some of my older posts Saas application, Web application with Spring boot security OR Spring boot application with docker. Even though none of these applications have Contact Us page, I will recommend to add that page with a simple form like below:

Contact Us

I have used Bootstrap template to build this form. This form is outside the web application, but I have similar form inside the web application for users to contact sales or support team. In this case, user who wants to sign up for the application can contact my sales team.

Now to use Spring provided facility for sending email from the application to your designated email, we will add following library:

compile('it.ozimov:spring-boot-email-core:0.6.3')

This library provides a EmailService which covers the spring library for spring-boot-starter-mail and do not have to write part of the code to send email. In this example, I will show this EmailService can be used to write a simple method to send email.

First we need to enable email tools by an annotation @EnableEmailTools in our main Spring application. Once we do that, I have written a simple method to send email. This method will look like below:

@Autowired
private EmailService emailService;

private void sendEmail(String emailAddress, String message, String phoneno) throws UnsupportedEncodingException, AddressException
{
    ArrayList<InternetAddress> emails = new ArrayList<>();
    emails.add(new InternetAddress("betterjavacode.com@gmail.com"));
    final Email email = DefaultEmail.builder()
            .from(new InternetAddress(emailAddress))
            .to(emails)
            .subject("Sales Support")
            .body(message + "\n" + phoneno)
            .encoding("UTF-8").build();

    emailService.send(email);
}

Now to make this email service to work, we still have to provide SMTP server properties and a from sender email and password. In this example above, I showed betterjavacode.com as my from gmail address.

Adding following properties in application.properties will set up our SMTP host for sending the email.

spring.mail.host = smtp.gmail.com
spring.mail.port = 587
spring.mail.username = betterjavacode.com@gmail.com
spring.mail.password =*****************
spring.mail.properties.mail.smtp.starttls.enable=true
spring.mail.properties.mail.smtp.starttls.required=true
spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.connectiontimeout=5000
spring.mail.properties.mail.smtp.timeout=5000
spring.mail.properties.mail.smtp.writetimeout=5000

Conclusion

In this post, I showed how to send email using the Spring boot email configuration feature.

 

How to use JProfiler for Performance testing

If you are a developer, some point of time you will have to use JProfiler for measuring the performance of your application. Many times, this is left to specialized performance teams or developers who know the tools well, but don’t know much about the code that they are trying to measure the performance. This approach, despite might be working, can have its flaws. Especially since performance team members are not writing the code. So as part of this past, I want to debunk few myths around this and why a developer should equally focus on a tool like JProfiler to check the performance.

What will you need

  • JProfiler (standalone or plugin in IntelliJ)
  • IntelliJ
  • A web application that you can profile

Understanding JProfiler

First JProfiler is a tool to understand what is going on inside a running JVM. Before we run the test, we should understand the basic functionality of JProfiler and how it will help us in performance improvement.

There are three aspects to JProfiler:

Time Profiling – This measures the execution paths of your application on the method level.

Memory Profiling – This provides the in-depth understand of heap usage by the application.

Thread Profiling – This analyses the thread synchronization issues.

JProfiler combines time, memory and thread profilers in a single application.

Performance Test

In this test, we will be profiling java code of the application. You can download JProfiler as a standalone tool to run or download a plugin with IntelliJ which is what I have done in my case.

Once you start IntelliJ for your web application either locally or remotely, you will see an icon on top right to start the JProfiler. If you are running the application locally, JProfiler will launch and will ask user to find the directory where JProfiler.exe is located, once you click OK, it will start the application server (from intelliJ RUN configuration).

JProfiler from IntelliJ

In Filter Settings , you can select the option Edit and it will launch another window where you can provide other application related settings. You can also run JProfiler for a remote application. For that you can provide the application’s host and port address in this application settings.

Application Settings

In my case, I am running the application locally, so I will skip these settings for right now.

Once the OK button is clicked, it will launch the tomcat webserver that I am using in this Spring based web application. I will launch my application in another browser window and do the necessary testing with application. JProfiler should profile my activity. Once you do that, we can verify CPU view for hot spots that Profiler found in our application. This will look like below:

Hot Spots

One good thing about this feature is that it shows where the application is taking more time and it is arranged in descending order. So most of the time that my code is taking in the libraries that I am using. If you expand each node, you will find in detail about each method that is getting called.

Conclusion

In this post, I showed how to use JProfiler. This was an introductory post as I have not talked about the different views that JProfiler offers for profiling the application.

Migrate Android Project from Eclipse to Android Studio

I remember downloading Android Studio back in 2013 to work on an Android project. But after playing around for an hour or two, I got frustrated. I had to delete Android Studio and go back to Eclipse. One reason why Eclipse was never successful with android because of patchy user interface for building Android applications, unfortunately that’s still the case in 2018.

Well, at least Google has made significant changes with Android Studio from 2015. It is almost compulsory to use Android Studio to build any new android applications. But there are few eclipse devotes like me who were still using eclipse to build android applications.

In this post, I will show how to migrate an android project from Eclipse to Android Studio. Warning: Even though the post looks very simple, it took me sizable efforts to figure out this migration. Hope it will be easier for readers.

Steps to migrate from Eclipse to Android Studio

  1. Download Android Studio from here. I had downloaded Android Studio version 3.3 RC2, but for it kept giving me syncing error when I imported the project. So I downgraded Android Studio version 3.2.1. Install Android Studio on your environment.
  2. On starting screen of Android Studio, you will see an option to import existing ADT project. Here you choose your eclipse based android project.
    1. Other option in this scenario, is to export android project from eclipse as gradle based project and it will also make life easier to import that project in Android Studio.
  3. Once you import the project in Android Studio, it will take some time to sync. In sync, Android Studio is basically creating gradle build file for your project. Converting all your library jars into dependencies. Other task it will be doing is to merge all your AndroidManifest.xml files. If there are any errors, it will show those errors during sync. Easiest way to fix AndroidManifest.xml errors are to go to this screen:

 4. Once the sync is successful, you can still face some issue to build the project. One major change with android studio gradle build is that in dependencies compile has been replaced by implementation project OR implementation files. Also to pull most of your dependencies, you will have to add google() as maven URL. minSdkVersion and targetSdkVersion should be moved to build file now and you can remove that from AndroidManifest.xml . Most of this should let you build your project successfully.

Conclusion

In this post, I showed how to migrate from eclipse to android studio for your android project. It is straight forward, but definitely depending on the project, you will run into few issues that you will have to resolve while building the project. Send me email or comment on this post if you have any questions.

 

 

Supporting FORMS authentication for ADFS in Android applications

Scenario with ADFS Authentication

Recently I came across this scenario where an application was using ADFS for authentication. The administrator had configured Windows Integrated Authentication as well as Forms Authentication as authentication policies. But when the user accessed the Android application, passive authentication through the Android webview browser fails because it redirects to the Windows Integrated Authentication URL instead of Forms Authentication. As the webview browser does not support WIA.

Android Phone

A solution to configure ADFS

Most organizations when using Active  Directory Federation Services (ADFS), use windows integrated authentication for their intranet. This helps users in their networks to enter credentials only once. Most browser-based applications provide a seamless experience to users without asking them to enter credentials again.

However, in this scenario, a user was trying to access an android application that equally needs passive authentication with an android webview browser. But android webview browser does not support WIA that results in authentication failure.

In such a scenario, ADFS should fall back to FORMS authentication and redirect the user to the login form. ADFS for windows server provides a simple configuration trick for authentication to fall back to FORMS.

  1. WIASupportedUserAgentString property of Set-ADFSProperties command
  2. WindowsIntegratedFallbackEnabled property of Set-AdfsGlobalAuthenticationPolicy command

Set-AdfsGlobalAuthenticationPolicy -WindowsIntegratedFallbackEnabled $true

WIASupportedUserAgentString gives the user agents that support WIA. If the component of the user agent string does not match any of the components of the user agent strings that are configured in WIASupportedUserAgentString property, ADFS will fall back to providing forms-based authentication, provided WindowsIntegratedFallbackEnabled flag is set to true.

Get-AdfsProperties | Select -ExpandProperty WIASupportedUserAgents

MSIE 6.0

MSIE 7.0; Windows NT

MSIE 8.0

MSIE 9.0

MSIE 10.0; Windows NT 6

MSIE 11.0; Windows NT 10

Windows NT 6.3; Trident/7.0

Windows NT 6.3; Win64; x64; Trident/7.0

Windows NT 6.3; WOW64; Trident/7.0

Windows NT 6.2; Trident/7.0

Windows NT 6.2; Win64; x64; Trident/7.0

Windows NT 6.2; WOW64; Trident/7.0

Windows NT 6.1; Trident/7.0

Windows NT 6.1; Win64; x64; Trident/7.0

Windows NT 6.1; WOW64; Trident/7.0

Windows NT 10.0; Trident/7.0

Windows NT 10.0; Win64; x64; Trident/7.0

Windows NT 10.0; WOW64; Trident/7.0

MSIPC

In this particular case, we removed Safari and Chrome from the list of user-agent strings, that’s when the authentication for application worked through passive authentication.

Conclusion

In conclusion, I showed in this post how to use FORMS authentication with ADFS for an android application. If you enjoyed this post, subscribe to my blog.

References

  1. Configuring Forms based authentication
  2. Forms based authentication