Hold on to it. This is going to be a rant about what I am thinking about ideas to post, but also an update about the web application Social KPI.
I am working on a few ideas that I would like to write about. But I am not sure. Lately I have been working on microservice architecture project and that had helped me to design Social KPI application. I would like to hear from my followers if they are interested in any particular topic that I should cover. Currently most of the interests are around Spring boot and microservices. That is too specific, but also equally too big of a topic to cover. I have covered bits and pieces of Spring boot.
Here are a few ideas that I have in mind that I would like to post about:
Spring boot in android applications.
How to use Spring boot and deploy in cloud infrastructure
What is chaos engineering?
Android application and details
Microservices and Service-to-Service authentication
Udemy course for Spring boot and microservices.
Please leave a comment if you want me to cover something new.
Where are we with Social KPI application?
So last, I posted about this application was back in May Twitter Client. After that there had been some progress as I did figure out how to use social login for authentication purposes, but it had not been added in the application. I will be reviving the work on this project and will try to contribute daily for 30-60 minutes. As part of my planning process, I will add here the tasks that need to be finished:
Add Social login UI for the application
Add UI Pages for displaying Social KPI reports and user navigation
Connecting front-end to back-end REST APIs through clients.
Use of Jasper reports for graphical reports.
These are the 4 big stories I am planning to finish by the end of October. Once I have all the code completed, I will launch the application through Heroku.
Java 8 added a new feature called Streams. Streams represent a sequence of objects from a source. In this post, I show how to use Streams API.
Previously, using collections API, we would have a collection of objects and then a developer would process this collection to manipulate further to query. With Streams feature, the developer will not have to do any processing operation over the collection of objects.
Streams
Firstly, streams provide a set of elements in a sequential manner. It provides a number of APIs for aggregate operation. Streams take Arrays, Collections, or I/O sources as an input.
How Streams Work
A stream represents a sequence of elements. Stream operations are either intermediate or terminal. Intermediate operations return streams to process further while terminal operations return either void or non-stream results.
As shown above, filter, map, sorted are intermediate operations and forEach is a terminal operation. Javadocs provide the list of all operations on streams.
Most stream operations accept some kind of lambda expression parameter, a functional interface specifying the behavior of the operation.
Instead of using collections, you can also use Stream.Of() operation to create a stream from a bunch of objects.
Intermediate operations have a characteristic of laziness. To look at this, let’s check the example below:
In the example shown above, I have list of strings that I filter to get only a list containing non-empty strings.
Streams also offer forEach operation which can be used to iterate over the elements of the stream.
Collect is a terminal operation that can transform the elements of a stream into a different kind of result. Example – a List, Map, or a Set.
Map is an operation that allows us to transform objects of a stream into another type of object.
Reduce operation combines all elements of the stream into a single result.
ParallelStreams
Streams also offer something called ParallelStreams . Basically all operations performed over ParallelStreams ,are performed in parallel. Depending on your usage, use these streams carefully as they can cause concurrency issues.
Example – listOfStrings.parallelStream()
ParallelStreams can be used to improve the runtime performance on a large set of input elements.
Conclusion
In this post, I showed how to use Streams API in Java 8. Streams make it handy in many cases where we have a list of objects and we are processing these objects. If you enjoyed this post, subscribe to my blog here.
So I have talked about blockchain previously here. I will not be indulging in details about what is blockchain. But I will show how to build blockchain using Spring boot.
What will you need
IntelliJ
Java 8
Spring boot
Gradle
What is blockchain?
In short, blockchain is chain of records which are called blocks, containing any kind of data or transactions in these records. They are chained together using hashes.
Build a blockchain
We will need two model classes: one each for block and for transaction. We will also need a spring rest controller to provide 3 APIs for mining, transaction, and chaining. The heart of this blockchain will be a utility class to provide us Proof of work. If you don’t know what is Proof of work, then you can revisit the article I have linked in this post where I explained the algorithm used in building a blockchain, is called as Proof of work.
Model Classes
Each block will contain an index, timestamp, transactions, proof, and a hash for previous block. This model class will look like below:
package com.betterjavacode.blockchain.model;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.hash.Hashing;
import java.nio.charset.StandardCharsets;
import java.util.List;
public class Block
{
public Block()
{
}
private Long index;
private Long timestamp;
private List<Transaction> transactionList;
private Long proof;
private String previousBlockHash;
public static final Long GENESIS_BLOCK_PROOF = 100L;
public static final String GENESIS_BLOCK_PREV_HASH = "1";
public Long getIndex()
{
return index;
}
public String getPreviousBlockHash()
{
return previousBlockHash;
}
public Long getProof()
{
return proof;
}
public List<Transaction> getTransactionList()
{
return transactionList;
}
public Block(Builder builder)
{
this.index = builder.index;
this.timestamp = builder.timestamp;
this.transactionList = builder.transactionList;
this.proof = builder.proof;
this.previousBlockHash = builder.previousBlockHash;
}
public static class Builder
{
private Long index;
private Long timestamp;
private List<Transaction> transactionList;
private Long proof;
private String previousBlockHash;
public Builder setIndex(Long index)
{
this.index = index;
return this;
}
public Builder setTimestamp(Long timestamp)
{
this.timestamp = timestamp;
return this;
}
public Builder setTransactionList(List<Transaction> transactionList)
{
this.transactionList = transactionList;
return this;
}
public Builder setProof(Long proof)
{
this.proof = proof;
return this;
}
public Builder setPreviousBlockHash(String previousBlockHash)
{
this.previousBlockHash = previousBlockHash;
return this;
}
public Block build()
{
return new Block(this);
}
}
public String hash(ObjectMapper mapper) throws JsonProcessingException
{
String json = mapper.writeValueAsString(this);
return Hashing.sha256().hashString(json, StandardCharsets.UTF_8).toString();
}
}
It will look like below when we have some transaction
We will write a rest controller to retrieve the chain, mine or add a transaction. A REST controller for transactions will do a POST request to add a transaction to chain. Our REST controller will look like below:
package com.betterjavacode.blockchain.controller;
import com.betterjavacode.blockchain.model.Block;
import com.betterjavacode.blockchain.model.Transaction;
import com.betterjavacode.blockchain.response.ChainResponse;
import com.betterjavacode.blockchain.response.MineResponse;
import com.betterjavacode.blockchain.response.TransactionResponse;
import com.betterjavacode.blockchain.service.Blockchain;
import com.betterjavacode.blockchain.util.BlockProofOfWorkGenerator;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import javax.validation.Valid;
import java.math.BigDecimal;
import java.util.UUID;
@RestController
@RequestMapping("/")
public class BlockchainController
{
@Autowired
private Blockchain blockchain;
@Autowired
private ObjectMapper objectMapper;
public static final String NODE_ID = UUID.randomUUID().toString().replace("-","");
public static final String NODE_ACCOUNT_ADDRESS = "0";
public static final BigDecimal MINING_CASH_REWARDS = BigDecimal.ONE;
@GetMapping("mine")
public MineResponse mine() throws JsonProcessingException
{
// Calculate Proof of work
Block lastBlock = blockchain.lastBlock();
Long lastProof = lastBlock.getProof();
Long proof = BlockProofOfWorkGenerator.proofOfWork(lastProof);
// Reward the miner by adding a transaction
blockchain.addTransaction(NODE_ACCOUNT_ADDRESS, NODE_ID, MINING_CASH_REWARDS);
// Add the new block to the chain
Block newBlock = blockchain.createBlock(proof, lastBlock.hash(objectMapper));
return new MineResponse.Builder().message("New Block Added").index(newBlock.getIndex()).transactions(newBlock.getTransactionList())
.proof(newBlock.getProof()).previousHash(newBlock.getPreviousBlockHash()).build();
}
@GetMapping("chain")
public ChainResponse fullChain()
{
return new ChainResponse.Builder().chain(blockchain.getChain()).length(blockchain.getChain().size()).build();
}
@PostMapping("transactions")
public TransactionResponse newTransaction(@RequestBody @Valid Transaction transaction)
{
Long index = blockchain.addTransaction(transaction.getSender(), transaction.getRecipient(), transaction.getAmount());
return new TransactionResponse.Builder().index(index).build();
}
}
Transaction POST call basically adds the transaction to the list and returns the index of the block to which the transaction will be added to.
Proof of work
We will discuss how we have implemented the proof of work here. This will be the heart of entire blockchain that we have built.
When new blocks are mined on the blockchain, a proof of work algorithm is used to verify if the block is justifiable. Simple idea of Proof of Work is to find a number which solves a problem. This number must be difficult to find, but easy to verify by network.
Example – Hash of an integer multiplied by another integer must end in a particular number. In our implementation for this algorithm, we verify the proof is valid as below:
public static boolean validProof(Long lastProof, Long proof)
{
String s = "" + lastProof + "" + proof;
String sha256 = Hashing.sha256().hashString(s, StandardCharsets.UTF_8).toString();
return sha256.endsWith(PROOF_OF_WORK);
}
This is the algorithm that miners try to solve fastest and whoever solves it first and compact manner gets rewarded with a coin as part of the transaction.
Swagger API
So now once we build our implementation and run the Spring Boot Application, I can access it through Swagger APIs. These APIs are as below:
/transactions – Creates a new transaction in the block
/mine – mines the new block
/chain – returns the full blockchain
Conclusion
In this post, we showed how to understand a blockchain by implementing one in Spring boot. Remember, one thing that I have not described is the consensus algorithm to verify the chain, though I have implemented that in my class Blockchain.
In post How to build docker image for your application, we saw how to create a docker container image for your application and then run that image. In this post, I want to revisit some of the docker commands and how to use this docker container in ECS (Elastic Container Service) which is offered by Amazon.
Publishing your docker container image in ECR (Elastic Container Repository) – Amazon offers a repository where you can push your docker image to store. Once you have the image in the repository, it can be used in ECS.
docker tag source_image:[tag] target_image:[tag]
docker push image:[tag] repository:image
2. You can pull this image from repository to run on your local as below
docker pull image:[tag]
3. Once you have docker image published in the repository, it is a straightforward. Create a cluster in ECS.
4. Cluster is nothing but collection of multiple services running on their own EC2 instances, provided you create a service of type EC2. Once you create a cluster, you can create a service. Another type of service you can create is that of Fargate. But we will not talk about it in this post.
5. When you create a service, it will ask you to enter the value for task, that means you have to create a task first. Enter the name and type of task (EC2).
6. Provide the docker container image and any environment variables you need to run this docker container. Provide necessary details of how much memory you need and if you are using storage.
7. Once you create a task, you can use it creating a service from Step 5. Choose an application load balancer in your service. If you haven’t already created load balancer and target group, you will need to create those. Use the newly created load balancer and target group for your service.
8. Now once everything is created, you can start the task to run. So this service will be available in the cloud. Amazon offers a healthcheck for your service, that you can configure while configuring service.
Conclusion –
In this post, I showed how to use Amazon ECS service for creating a service and running that service in Amazon cloud. If you want to learn about docker containers, learn here.
In this post, I will show how to use social login in a Spring Boot application. So we build an application, but we use a form-based login which is the most basic and most insecure authentication mechanism out there. How do we get over this hunch and use the latest more secure mechanism?
Social login – Tada.
Yes, with an increasing number of social networks, it has become increasingly popular and easier to build an OAuth based login mechanism using social networks. In other words, spring boot offers a solution with a social login plugin and in this post, we will show how to use social login to authenticate your users.
What will you need
IntelliJ
Java 8
Twitter/Facebook/Google/Linkedin/Github accounts
Spring Boot
Gradle
Spring Social Core
Spring offers a spring-social-core project that contains APIs to connect to user’s social accounts. Nevertheless, this library includes a connect framework that offers a solution to manage connections with social service providers. It offers support for OAuth1a and OAuth2. The simplest way to understand this library is that you create a connection factory for each social provider. A connection factory locator finds a factory to create a Sign In Provider. I will provide more details as we go along in implementing this module.
Create a Social Login Gradle Project
If you haven’t noticed from my blog posts, but I have switched from eclipse to IntelliJ for programming editor. Intellij is just smarter and easy to write code editor. So first create a Gradle project for spring boot. (Side note – if you are using IntelliJ ultimate edition, it offers a feature to create spring project.) We will be using the latest version of Spring Boot (2.0.3.RELEASE) to build this project.
I will explain each dependency added in Gradle file as we go along.
Create an entity class
We will be using a simple entity class for User with just one field name. This will look like below:
@JsonIgnoreProperties(ignoreUnknown = true)
public class User
{
public String name;
public String getName()
{
return name;
}
public void setName(String name)
{
this.name = name;
}
}
Create A Social Configuration to adapt Spring Social library
Firstly, we will implement an interface SocialConfigurer that Spring social library offers. For instance, as part of this implementation, we will create connection factories for different social service providers. Also for this module, we are using InMemoryUsersConnectionRepository. You can always implement a JDBC based database user connection repository. This class will look like below:
package com.betterjavacode.reusablesociallogin;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.*;
import org.springframework.core.env.Environment;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.crypto.encrypt.Encryptors;
import org.springframework.security.web.savedrequest.HttpSessionRequestCache;
import org.springframework.social.UserIdSource;
import org.springframework.social.config.annotation.ConnectionFactoryConfigurer;
import org.springframework.social.config.annotation.EnableSocial;
import org.springframework.social.config.annotation.SocialConfigurer;
import org.springframework.social.config.annotation.SocialConfigurerAdapter;
import org.springframework.social.connect.Connection;
import org.springframework.social.connect.ConnectionFactoryLocator;
import org.springframework.social.connect.ConnectionRepository;
import org.springframework.social.connect.UsersConnectionRepository;
import org.springframework.social.connect.jdbc.JdbcUsersConnectionRepository;
import org.springframework.social.connect.mem.InMemoryUsersConnectionRepository;
import org.springframework.social.connect.support.ConnectionFactoryRegistry;
import org.springframework.social.connect.web.ProviderSignInController;
import org.springframework.social.facebook.connect.FacebookConnectionFactory;
import org.springframework.social.github.connect.GitHubConnectionFactory;
import org.springframework.social.google.connect.GoogleConnectionFactory;
import org.springframework.social.linkedin.connect.LinkedInConnectionFactory;
import org.springframework.social.security.AuthenticationNameUserIdSource;
import org.springframework.social.twitter.api.Twitter;
import org.springframework.social.twitter.api.impl.TwitterTemplate;
import org.springframework.social.twitter.connect.TwitterConnectionFactory;
import javax.inject.Inject;
import javax.sql.DataSource;
@Configuration
@PropertySource("classpath:application.properties")
@EnableSocial
public class SocialConfig implements SocialConfigurer
{
@Autowired
private DataSource dataSource;
@Override
public void addConnectionFactories(ConnectionFactoryConfigurer connectionFactoryConfigurer, Environment environment)
{
connectionFactoryConfigurer.addConnectionFactory(new TwitterConnectionFactory(environment.getProperty("spring.social.twitter.consumerKey"), environment.getProperty("spring.social.twitter.consumerSecret")));
connectionFactoryConfigurer.addConnectionFactory(new FacebookConnectionFactory(environment.getProperty("spring.social.facebook.appId"),environment.getProperty("spring.social.facebook.appSecret")));
GoogleConnectionFactory googleConnectionFactory = new GoogleConnectionFactory(environment.getProperty("spring.social.google.appId"),environment.getProperty("spring.social.google.appSecret"));
googleConnectionFactory.setScope("profile");
connectionFactoryConfigurer.addConnectionFactory(googleConnectionFactory);
connectionFactoryConfigurer.addConnectionFactory(new GitHubConnectionFactory(environment.getProperty("spring.social.github.appId"), environment.getProperty("spring.social.github.appSecret")));
connectionFactoryConfigurer.addConnectionFactory(new LinkedInConnectionFactory(environment.getProperty("spring.social.linkedin.appId"), environment.getProperty("spring.social.linkedin.appSecret")));
}
@Override
public UserIdSource getUserIdSource()
{
return new UserIdSource() {
@Override
public String getUserId() {
Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
if (authentication == null) {
throw new IllegalStateException("Unable to get a ConnectionRepository: no user signed in");
}
return authentication.getName();
}
};
}
@Override
public UsersConnectionRepository getUsersConnectionRepository(ConnectionFactoryLocator connectionFactoryLocator)
{
InMemoryUsersConnectionRepository usersConnectionRepository = new InMemoryUsersConnectionRepository(
connectionFactoryLocator);
return usersConnectionRepository;
}
}
As you see in this class, I am referring to application.properties . The application.properties will look like below:
In other words, to get clientid and clientsecret , you will have to register your application with each social service provider. We will not be covering that in this post.
Create a spring web security configuration
In this class, we will extend websecurityconfigureradapter and configure HTTP security as part of spring security implementation. We also add a bean to create Sign In Providers which are part of Spring Social. In addition, we will implement this Sign In Provider to provide a facility to users to sign in with their social provider.
As you see in this class, we have a bean ProviderSignInController which will use SocialSignInAdapter.
Implement a Sign In Adapter
Above all, this is the heart of our implementation where authentication will take place and the user will be assigned a role to access the application. The user will be redirected to the application if the user successfully authenticates. This class will look like below:
As you see in getAuthentication, we pass userId and roles for token-based authentication.
If the user has not signed up with a social provider before, he will be asked to sign up and will be redirected to the application after the first time sign up.
package com.betterjavacode.reusablesociallogin.social;
import com.betterjavacode.reusablesociallogin.entity.User;
import com.betterjavacode.reusablesociallogin.util.UserHelper;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.social.connect.Connection;
import org.springframework.social.connect.ConnectionSignUp;
import org.springframework.stereotype.Service;
@Service
public class SocialConnectionSignup implements ConnectionSignUp
{
@Autowired
UserHelper userHelper;
@Override
public String execute(Connection<?> connection)
{
User user = userHelper.getUser(connection);
return user.getName();
}
}
As you see in this class, we have Autowired a userHelper class, this class will have an implementation to get user details from each social provider.
Therefore, this UserHelper will look like below:
package com.betterjavacode.reusablesociallogin.util;
import com.betterjavacode.reusablesociallogin.entity.User;
import org.springframework.social.connect.Connection;
import org.springframework.social.facebook.api.Facebook;
import org.springframework.social.github.api.GitHub;
import org.springframework.social.google.api.Google;
import org.springframework.social.linkedin.api.LinkedIn;
import org.springframework.social.twitter.api.Twitter;
import org.springframework.stereotype.Component;
@Component
public class UserHelper
{
public User getUser(Connection<?> connection)
{
User user = null;
//get the connection type
ConnectionType type = ConnectionHelper.getConnectionType(connection);
if (type.equals(ConnectionType.TWITTER)) {
user = getTwitterUser(connection);
} else if (type.equals(ConnectionType.FACEBOOK)) {
user = getFacebookUser(connection);
} else if (type.equals(ConnectionType.GOOGLE)) {
user = getGoogleUser(connection);
} else if (type.equals(ConnectionType.GITHUB)) {
user = getGithubUser(connection);
} else if (type.equals(ConnectionType.LINKEDIN)){
user = getLinkedInUser(connection);
}
return user;
}
private User getTwitterUser(Connection<?> connection)
{
User user = new User();
Twitter twitterApi = (Twitter)connection.getApi();
String name = twitterApi.userOperations().getUserProfile().getName();
user.setName(name);
return user;
}
private User getFacebookUser(Connection<?> connection)
{
User user = new User();
Facebook facebookApi = (Facebook)connection.getApi();
String [] fields = { "name" };
User userProfile = facebookApi.fetchObject("me", User.class, fields);
String name = userProfile.getName();
user.setName(name);
return user;
}
private User getGoogleUser(Connection<?> connection)
{
User user = new User();
Google googleApi = (Google) connection.getApi();
String name = googleApi.plusOperations().getGoogleProfile().getDisplayName();
user.setName(name);
return user;
}
private User getGithubUser(Connection<?> connection)
{
User user = new User();
GitHub githubApi = (GitHub) connection.getApi();
String name = githubApi.userOperations().getUserProfile().getName();
user.setName(name);
return user;
}
private User getLinkedInUser(Connection<?> connection)
{
User user = new User();
LinkedIn linkedInApi = (LinkedIn) connection.getApi();
String name = linkedInApi.profileOperations().getUserProfile().getFirstName();
user.setName(name);
return user;
}
}
Implementing a controller and views
Similarly, the last piece in this puzzle is to add a controller and corresponding views so when the user accesses the application, the user will be challenged for authentication.
However, we will add a login controller which will have three views for login, sociallogin and socialloginsuccess . This will look like below:
@Controller
public class LoginController
{
@RequestMapping(value="/login", method= RequestMethod.GET)
public String login(Model model)
{
return "login";
}
@RequestMapping(value ="/socialloginhome", method = RequestMethod.GET)
public String socialloginhome(Model model)
{
return "socialloginhome";
}
@RequestMapping(value="/socialloginsuccess", method= RequestMethod.GET)
public String socialloginsuccess(Model model)
{
return "socialloginsuccess";
}
}
Running the application
Once I build the application and run it, the flow will look like below:
You click on hereit will take you to social login screen as below:
I will choose Facebook and server-side code will redirect me to Facebook login screen. Once I enter my credentials, Facebook will redirect back me to my application as below:
Hence, we showed successful social authentication. Lastly, social login is part of any saas application you are building.
Conclusion
In conclusion, we showed how to create a social login module using the Spring boot social feature. Moreover, the code for this will be available to download here.