Category Archives: Microservices

How to integrate reCaptcha with Spring Boot Application

In this post, I want to show how to integrate ReCaptcha in a Spring Boot application. This will be an important step if you have forms in your application and if those forms are publicly available on the internet. You can face a lot of spams or bots trying to fill those forms. To avoid these spams from bots, ReCaptcha is of utmost importance.

Google offers a reCaptcha service that we will integrate with Spring Boot application to stop bots from submitting the forms in our application.

Registration with Google for ReCaptcha

Google offers a reCaptcha service that developers can use in their applications. As part of this implementation, we will register our service in Google APIs so that Google can provide us credentials to use while calling its service.

We can register our site at Google Recaptcha Administration. This registration will provide us with site-key and site-secret.

Now as part of Spring boot application, we can store these credentials in application.properties file as below:

# ====================================================================================
# Google reCaptcha Settings
# ====================================================================================
google.recaptcha.key.site=site-key
google.recaptcha.key.secret=site-secret

We will make these properties available through a spring bean object in our application, so we can use them when we call Google Recaptcha Service.

UI changes to display ReCaptcha box

Now to display Recaptcha box on your form, we will add the following code in our templates where our public form resides. In this case, I am putting Recaptcha box in Contact Us and Sign Up pages as those are publicly available forms.

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:th="http://www.thymeleaf.org">

<head>
    <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" />
    <title>Renters Feedback</title>

    <!-- other scripts and stylesheets -->
    <script src='https://www.google.com/recaptcha/api.js'></script>

</head>
<body>

<form method = "post" id="contactus" th:action="@{/contact}">
<!--- other code to display contact us form -->
<div class="g-recaptcha" th:attr="data-sitekey=${@captchaSettings.getSite()}"></div><br/><br/>
<span id="captchaError" class="alert alert-danger col-sm-4" style="display:none"></span>
</form>

</body>
</html>

 

Server-side handling of ReCaptcha

So as we have a widget on our form. When a user completes a challenge and submits the form, the request will be sent to the server containing an encoded site key and a unique string that identifies user’s challenge completion.

But on the server side, we can’t just assume and trust what the user has submitted. So we need to verify this captcha challenge by sending a request to Google API at https://www.google.com/recaptcha/api/siteverify by passing captcha response we have received.

If the verification is successful, the json response from API will contain a success parameter.

If the verification fails, then an application will throw an exception and reCaptcha library will instruct a client to create a new challenge.

One thing to understand here is that we limit how many attempts a client can make to reCaptcha challenge. The reason for this is to avoid any kind of DoS attack. Of course, this is precautionary and elementary step. We implement ReCaptchaAttemptService to block if a client tries to use the challenge for more than 4 attempts.

Demo of the complete workflow of ReCaptcha

Conclusion

In this post, we showed how we can use google reCaptcha service to integrate a reCaptcha widget in a form that is publicly available to avoid bots spams. From the security perspective, this is an important step and developers should take into account this important feature for their web applications.

References

  1. Recaptcha with Spring Boot Application – Recaptcha With Spring Boot Application
  2. Protecting Spring Boot application with Google Recaptcha – Protecting Spring Boot Application

 

 

Techstack Framework for RentersFeedback

As a developer, we make different choices based on what is available to us and what we know. But are those choices always better? They may not or they may. It really depends. In this post, I describe the techstack framework that I used to build Renters Feedback.

You can read my post, how I came up with an idea to build an application for renters feedback.

I wanted to describe the process of the choices I made to choose a tech stack framework for building the application. Considering my expertise in Spring Boot, it was a default choice to use to build this application. There are other factors I took into account like the ease of coding, ease of deploying as a microservice-based application in the cloud, and docker.

Techstack Framework for Renters Feedback

Development Framework For Renters Feedback

For developing the application, my focus was on the re-usability of code. Since I have written few applications as part of this blog using Spring Boot, there was authentication, login forms, sign up forms, most of that code was readily available. One thing I have to think through for RentersFeedback was database modeling.

Database Framework for Renters Feedback

For developing a database based application, I used mysql as a development database. In production, I changed that to postgresql . 

Why database change? 

Heroku support for mysql wasn’t straightforward, so I preferred what was available by default and it was postgresql. I have to do a few changes to mysql scripts. I could have automated these scripts through liquibase, but I preferred not to for the first version of the product. In the future, when I plan to add more changes to the database model, I will add liquibase-based scripts.

Authentication Scheme

There were some questions about why a user needs to login on a RentersFeedback website. Well, someone has to post those reviews before people can browse it. You need to login if you want to post a review.

The easiest choice was to have form-based login even though it is getting old and not safe. I still feel most users would use email to login. User passwords are stored in an encrypted and hashed format in the database.

Another mechanism, I decided to add, was OAuth2 OpenId protocol by using Google API for the same. It was easy to implement and something I have expertise in.

I could add other social logins, but I prefer to keep it simple and if the need arises, I will add those logins in the future.

Using Google API, made me use Redis Cache. I wasn’t planning to use cache since the application is still in nascent stages, but now it is there, so future scaling would be easier from a performance perspective.

User Interface

The user interface was built using Spring Boot provided thymeleaf templates along with Twitter’s Bootstrap CSS library and javascript library. For the search feature, I have used javascript library of Google search APIs.

After deploying the application on Heroku, I came across a few issues about having forms available publicly. To avoid spams, I will be adding Captcha on those forms. I will show how to use reCaptcha APIs in the next post.

To allow users to reset the password, I have used Spring Boot Email system. This was an easy implementation once you know how the forgot your password flow works.

Deployment Environment

I used Heroku to deploy my application. Heroku has great documentation. It’s very easy to sync up with your GitHub repository. So if you push your changes to GitHub, it will sync up to build and deploy on Heroku.

Questions

Choosing the right tech stack framework for your application can be a difficult task if you are a beginner.  Since I have experience in building applications, choosing this techstack framework for Renters Feedback was a straightforward choice. If you have any questions about implementation, why I used certain technology, and how it can be improved, you can post a comment on this blog and I will answer those questions.

 

How to add Stripe Payment to Spring Boot Application

In this post, we will show how to add Stripe Payment to Spring boot application. Most enterprise applications offer a way where customer can pay online. Online payments are the backbone of internet world in current times. If you ever built an application for a customer, previously there were physical cheques OR credit card payments. With the applications becoming more web based, it has become utmost necessary to integrate your application with some kind of payment gateway. Payment gateway will handle all tax and financial regulation related complications which the application doesn’t have to deal with.

The functionality is part of the application Social KPI that I am building.

What are we trying to achieve here?

Story for Stripe Payment

An administrator comes on the billing page and pays the bill for that month. Once the bill is paid, the billing page will show that the bill has been paid for that month. So the option to enter credit card details will only be shown if the bill has not been paid.

As part of payment service, we will be using Stripe . Once the user enters credit card details and she can enter Pay Now button which will contact Stripe API to get token, this token will be used to create a charge on Stripe and Stripe will respond with success or failure of the charge.

Flow

To summarize the flow

  1. User clicks Pay Now to pay the charges
  2. Frontend Stripe javascript API contacts Stripe to create token using enter billing details
  3. Frontend Stripe javascript API sends this token to server to handle billing on server side
  4. On server side, controller uses the token and amount to create charge for that customer for application usage.
  5. For paid bills, the status is marked as paid for the customer for that month.

Frontend Implementation

To use Stripe APIs, we must first create account on stripe.com as a developer and get the keys. Good thing is Stripe offers API keys for test and live environments. For this post and demo, we will be using test keys only. Once we have API keys, we will use them in our frontend and backend implementation.

In following screenshot, you will see how the billing page will look:

Billing Page

Once the user clicks on Pay Now, the javascript function from Stripe for mounting card and creating token will be called. Once the token is available, the same function will pass it server by submitting a POST request. Sample of this code will look like below:


            var form = document.getElementById('payment-form');
            form.addEventListener('submit',function(event){
                event.preventDefault();
                payTheBill();
            });

            function payTheBill(){
                stripe.createToken(card).then(function(result){
                    if(result.error){
                        var errorElement = document.getElementById('card-errors');
                        errorElement.textContent = result.error.message;
                    } else {
                        var token = result.token.id;
                        var email = $('#email').val();
                        var companyid = $('#companyid').val();
                        var amount = $('#amount').val();
                        $.get(
                            "/createcharge",
                            {email:email,token:token,companyid : companyid, amount:amount},
                            function(data){
                                alert(data.details);
                            },'json');
                    }
                })
            }

Backend Implementation

As part of the application Social KPI, I have a database table billing to track customer’s paid bills. The PaymentController is a REST controller which will have a POST request mapping for creating a charge and storing in the database table and mark the bill as paid. As shown above in javascript code, once the token is available it will be sent to server side to controller to handle the charge. This will be a REST call, so the controller is also RestController.


 @RequestMapping(value="/createcharge",method = RequestMethod.GET)
    @ResponseBody
    public Response createCharge(String email, String token, String companyid, double amount)
    {
        LOGGER.info("Enter >> createCharge() ");

        if(token == null)
        {
            throw new RuntimeException("Can't create a charge, try again");
        }

        Billing billing = billingRepository.findByCompanyId(Integer.parseInt(companyid));

        double billedAmount = amount * 100;

        String chargeId = paymentService.createCharge(email,token,billedAmount);

        if(chargeId != null && !chargeId.equals(""))
        {
            LOGGER.info("bill has been charged on consumer's account");
            billing.setStatus(true);
            billing.setPaiddate(new Date());
            billingRepository.save(billing);
        }

        LOGGER.info("Exit << createCharge() ");
        return new Response(true,"Congratulations, your card has been charged, chargeId= "+chargeId);
    }

As shown above, Service called paymentService will create a charge on Stripe. To implement paymentService, you will need to include stripe java library in your implementation.

compile('com.stripe:stripe-java:10.5.0')

So the service class PaymentService will look like below to create charge:


    public String createCharge(String email, String token, double amount)
    {
        String id = null;
        try
        {
            Stripe.apiKey = API_SECRET_KEY;
            Map chargeParams = new HashMap<>();
            chargeParams.put("amount", (int)(amount*100));
            chargeParams.put("currency", "USD");
            chargeParams.put("description", "Charge for " + email);
            chargeParams.put("source", token);
            Charge charge = Charge.create(chargeParams);
            id = charge.getId();
        }
        catch(StripeException e)
        {
            throw new RuntimeException("Unable to process the charge", e);
        }
        return id;
    }

Once the bill is paid, administrator will see this as the bill paid for that month.

Conclusion

In this post, we showed how to integrate Spring boot application with Stripe payment gateway.

References

  1. Stripe API reference - https://stripe.com/docs/api/charges
  2. Integrate Stripe with Spring boot - Stripe with Spring boot

How to file upload using Spring Boot

In this post, I will show how I added file upload functionality in my Spring Boot application, social KPI.

On the outskirts, it looks very simple functionality and it is indeed simple with Spring Boot. As part of this post, we will build a web form where an administrator will add additional users for his/her company by importing a CSV file in a particular format.

Basic functionality is to provide a way for an administrator to import a CSV file, read and validate the data, and save it in the database if proper data.

Now once we have defined our user story, let’s get started with the post.

Form For File Upload In a Spring Boot Application

We are using thymeleaf templates for our spring boot based application. So writing a simple html page with a form to upload a file is very straight forward as below:

<div class="container importuser">
    <div class="form-group">
    <form method="POST" th:action="@{/uploadUsers}" enctype="multipart/form-data">
        <input type="hidden" name="companyGuid" th:value="${companyGuid}"/>
        <input type="file" name="file"/></br></br>
        <button type="submit" class="btn btn-primary btn-lg" value="Import">Import
        </button>
    </form>
    </div>
</div>

As you see in this form, clicking on Import button will kick the action to upload users.

Controller to handle file upload on backend side

A controller to handle upload users functionality will look like below:

    @RequestMapping(value = "/uploadUsers",method= RequestMethod.POST)
    public String fileUpload (@RequestParam("file") MultipartFile file, @RequestParam(
            "companyGuid") String companyGuid,
                              RedirectAttributes redirectAttributes)
    {
        LOGGER.info("File is {}", file.getName());
        LOGGER.info("Company Guid is {}", companyGuid);

        if (file.isEmpty())
        {
            redirectAttributes.addFlashAttribute("message", "Please select a file to upload");
            return "redirect:/uploadStatus";
        }

        List userList = FileUtil.readAndValidateFile(file, roleRepository);
        for(User user: userList)
        {
            User createdUser = userManager.createUser(companyGuid, user);
        }

        redirectAttributes.addFlashAttribute("message",
                "You successfully uploaded " + file.getOriginalFilename() + " and added " + userList.size() + " users");


        return "redirect:/uploadStatus";
    }

The method to readAndValidateFile is simply reading the data from file, validating to make sure all the fields in CSV file exists, if wrong format, it will throw an error. If a valid file, it will create a list of users. UserManager will create each user.

The class FileUtil is as below:

package com.betterjavacode.socialpie.utils;

import com.betterjavacode.socialpie.models.Role;
import com.betterjavacode.socialpie.models.User;

import com.betterjavacode.socialpie.repositories.RoleRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.multipart.MultipartFile;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.List;

public class FileUtil
{
    private static final String FIRST_NAME = "firstname";
    private static final String LAST_NAME = "lastname";


    public static List readAndValidateFile (MultipartFile file, RoleRepository roleRepository)
    {
        BufferedReader bufferedReader;
        List result = new ArrayList<>();
        try
        {
            String line;
            InputStream inputStream = file.getInputStream();
            bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
            while((line = bufferedReader.readLine()) != null)
            {
                String[] userData = line.split(",");
                if(userData == null || userData.length != 5)
                {
                    throw new RuntimeException("File data not in correct format");
                }
                if(FIRST_NAME.equalsIgnoreCase(userData[0]) && LAST_NAME.equalsIgnoreCase(userData[2]))
                {
                    continue; // first line is header
                }
                User user = new User();
                user.setFirstName(userData[0]);
                user.setMiddleName(userData[1]);
                user.setLastName(userData[2]);
                user.setEmail(userData[3]);
                Role role = roleRepository.findByRoleName(userData[4]);
                user.setRole(role);
                result.add(user);
            }
        }
        catch(IOException e)
        {
            throw new RuntimeException("Unable to open the file " + e.getMessage());
        }
        return result;
    }
}

A working demo

Once I log into the application Social KPI, I click on Add Users and it will take me to upload the users screen which will look below:

Spring Boot File Upload

Import Users

Once you choose a file in CSV format to upload and click on Import, it will show the screen as below:

Spring Boot File Upload - Upload Status

File Upload Status

Conclusion

So in this post, we showed how to import a file while using Spring Boot multipart form.

References

  1. Uploading files – uploading files

Monitoring your microservice with Micrometer

Spring Boot has made building a web application way easier.  It has also added a lot of other critical libraries that help enterprise applications in different ways. With enterprise applications moving to the cloud, Spring Boot has made it easier to deploy spring applications in the cloud with continuous integration. In this post, I will show how we can use a spring micrometer library to gather analytics related to your code.

As a result, these analytics can be transferred to different vendor databases for creating metrics-based dashboards. I showed how to use spring-boot-actuator to collect some metrics data.

As Spring defines Micrometer is a dimensional-first metrics collection facade. In simple words, it is similar to SLF4J, except for metrics.

Configure Micrometer for microservice

Firstly to use a micrometer, I have created a simple microservice with REST APIs and it is built using Spring Boot 2. Most importantly Spring Boot has added backward compatibility for Spring 1.x.

You can configure Micrometer in your Spring Boot 2.X based Microservice by adding the following dependency in your build file

runtime('io.micrometer:micrometer-registry-prometheus:1.0.4')

Adding Metrics

We will discuss different metrics that we can add through the micrometer. Dimensions and names identify a meter. You can use Meter for different types of metrics.

Counter

Counters are a cumulative metric. These are mostly used to count the number of requests, number of errors, number of tasks completed.

Gauges

A gauge represents a single value that can go up and down. The gauge measures memory usage.

Timers

Timers measure the rate at which we call a particular code or method. Subsequently we can also find out latencies when the execution of code is complete.

We talked about different metrics and how we can configure micrometers. Now we will show how to use this library to configure against a monitoring system. Spring micrometer supports the number of the monitoring system. In this post, I will be showing how to use against Prometheus monitoring system.

What is Prometheus?

Prometheus is an in-memory dimensional time-series database with a built-in UI, a custom query language, and math operations. To know more, you can visit here.

Meanwhile, we can add Prometheus in our microservice by adding the following dependency in the Gradle file

compile('org.springframework.boot:spring-boot-starter-actuator:2.0.3.RELEASE')
runtime('io.micrometer:micrometer-registry-prometheus:1.0.4')

For example, to understand where Prometheus lies in whole architecture, look at the below

Spring Boot microservice -> Spring Micrometer -> Prometheus

Once the above dependencies are added, Spring boot will automatically configure PrometheusMeterRegistry and CollectorRegistry to collect and export metrics data in a suitable format that Prometheus can scrape.

To enable Prometheus endpoints

Similarly, you enable Prometheus and actuator endpoints. Add following properties in application.properties file

management.security.enabled = false
management.endpoints.web.exposure.include=health,info,prometheus

Now if we run to start our webserver to see how these endpoints look, we can verify by going to endpoints http://localhost:8080/actuator/info , http://localhost:8080/actuator/health and http://localhost:8080/actuator/prometheus .  Prometheus endpoint looks like below :

Prometheus

Conclusion

In this post, we showed how to use Spring Micrometer to capture metrics data and configure with Prometheus. In the next post, I will show how to display this data in the human-readable format in nice UI using Prometheus.

References

  1. Production-Ready Metrics – Metrics
  2. Spring Micrometer – Spring Micrometer