Category Archives: REST

Database and design discussion – Part III

To continue the development of a spring-based web application, this post will discuss using of Twitter API in saas application. If you want to understand, what we are building, you can read the first two posts of this series where we discussed the design of the application we are building:

  1. Database design and discussion – Part I
  2. Database design and discussion – Part II

In the previous post, we discussed the Instagram API that we will be using. With recent events around Facebook, I have decided not to use Facebook API for application development. We will still use Instagram and Twitter API.

Using Twitter API in SAAS Application

Firstly, Twitter offers different APIs for developers to build applications. We will be using Engagement API. You can find more details Twitter API.

Our goal is to use this Twitter API to collect engagement metrics in the Saas application.

Secondly, Engagement API offers us details about account engagement metrics which can help us to design marketing strategy. Sample response of this API looks like the below:

{
  "Tweet metrics": {
    "902301386880286721": {
      "engagements": "433",
      "favorites": "21",
      "impressions": "72218"
    },
    "902731270274166784": {
      "engagements": "61",
      "favorites": "27",
      "impressions": "7827"
    },
    "907022936564838401": {
      "engagements": "187",
      "favorites": "37",
      "impressions": "1916"
    }
  }
}

Therefore this API provides metrics for tweets, which tweet generated more traffic. The key decision that can be devised based on these metrics is what kind of tweet, moment or incident generates the traffic.

What fields we will use in our database?

In conclusion, we will be using the following fields in our database table TwitterData

  • tweet id
  • engagements
  • impressions
  • tweet

Twitter is a viable medium. This data will provide small businesses with a key metric about what tweets have worked with their followers and how they can leverage that pattern. Once the small businesses sort out these patterns, they will be able to create a number of tweets to engage with customers. Eventually, the goal here is to help small businesses to attract customers, and repeat customers.

References

  1. Twitter API documentation – Twitter API

 

Redis Caching with RedisCacheManager

Introduction

In the previous post Redis Caching, we saw how to use Redis caching with all default settings. We didn’t have any Cache Manager or anything, but we were able to cache data. In this post, we will show how to use RedisCacheManager to cache the data. This manager can further be extended to customize the caching configuration even more. But we will not be looking into customization in this post particularly.

RedisCacheManager with Redis

Implement CacheManager for RedisCacheManager

Most of the code for this post will be similar to what we implemented in the previous post. We will just show how to use CacheManager.

To implement CacheManager first we remove @EnableCaching annotation from the main class SpringAppCacheApplication. Now we add a new CacheConfig class to configure our cache manager.

Basically, this CacheConfig will define CacheManager which build a redisTemplate to get JedisConnectionFactory which will be our java client to connect to our Redis server. This JedisConnectionFactory will get server host and port properties from application.properties file. The source code will look like below:

package com.betterjavacode.config;


import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cache.CacheManager;
import org.springframework.cache.annotation.CachingConfigurerSupport;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;
import org.springframework.context.support.PropertySourcesPlaceholderConfigurer;
import org.springframework.data.redis.cache.RedisCacheManager;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.connection.jedis.JedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;

@Configuration
@EnableCaching
@ComponentScan("com.betterjavacode.config")
@PropertySource("classpath:/application.properties")
public class CacheConfig extends CachingConfigurerSupport
{
    private static final Logger LOGGER = LoggerFactory.getLogger(CacheConfig.class);
    private @Value("${spring.redis.host}") String redisHost;
    private @Value("${spring.redis.port}") int redisPort;

    @Bean
    public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
        return new PropertySourcesPlaceholderConfigurer();
    }

    @Bean
    public JedisConnectionFactory redisConnectionFactory()
    {
        LOGGER.info(" Inside redisConnectionFactory()...");

        JedisConnectionFactory redisConnectionFactory = new JedisConnectionFactory();

        redisConnectionFactory.setHostName(redisHost);
        redisConnectionFactory.setPort(redisPort);
        redisConnectionFactory.setUsePool(true);
        return redisConnectionFactory;
    }

    @Bean
    public RedisTemplate<String, String> redisTemplate(RedisConnectionFactory rf)
    {
        LOGGER.info(" Inside redisTemplate()...");

        RedisTemplate<String, String> redisTemplate = new RedisTemplate<>();
        redisTemplate.setConnectionFactory(redisConnectionFactory());
        return redisTemplate;
    }

    @Bean
    public CacheManager cacheManager(RedisTemplate redisTemplate)
    {
        LOGGER.info(" Inside cacheManager()...");
        RedisCacheManager cacheManager = new RedisCacheManager(redisTemplate);
        cacheManager.setDefaultExpiration(300);
        return cacheManager;
    }
}

Now if we build our application and run it, Spring boot console will show the following output

2018-02-28 20:31:41.913  INFO 9856 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'requestContextFilter' to: [/*]
2018-02-28 20:31:42.034  INFO 9856 --- [           main] o.s.j.d.DriverManagerDataSource          : Loaded JDBC driver:com.mysql.jdbc.Driver
2018-02-28 20:31:42.244  INFO 9856 --- [           main] j.LocalContainerEntityManagerFactoryBean : Building JPA container EntityManagerFactory for persistence unit 'default'
2018-02-28 20:31:42.288  INFO 9856 --- [           main] o.hibernate.jpa.internal.util.LogHelper  : HHH000204: Processing PersistenceUnitInfo [ name: default        ...]
2018-02-28 20:31:42.495  INFO 9856 --- [           main] org.hibernate.Version                    : HHH000412: Hibernate
 Core {5.2.13.Final}
2018-02-28 20:31:42.499  INFO 9856 --- [           main] org.hibernate.cfg.Environment            : HHH000206: hibernate
.properties not found
2018-02-28 20:31:42.599  INFO 9856 --- [           main] o.hibernate.annotations.common.Version   : HCANN000001: Hiberna
te Commons Annotations {5.0.1.Final}
2018-02-28 20:31:43.688  INFO 9856 --- [           main] org.hibernate.dialect.Dialect            : HHH000400: Using dia
lect: org.hibernate.dialect.MySQL5Dialect
2018-02-28 20:31:43.764  INFO 9856 --- [           main] o.h.e.j.e.i.LobCreatorBuilderImpl        : HHH000423: Disabling
 contextual LOB creation as JDBC driver reported JDBC version [3] less than 4
2018-02-28 20:31:44.684  INFO 9856 --- [           main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA Enti
tyManagerFactory for persistence unit 'default'
2018-02-28 20:31:45.184  INFO 9856 --- [           main] com.betterjavacode.config.CacheConfig    :  Inside redisConnectionFactory()...
2018-02-28 20:31:45.288  INFO 9856 --- [           main] com.betterjavacode.config.CacheConfig    :  Inside redisTemplate()...
2018-02-28 20:31:45.346  INFO 9856 --- [           main] com.betterjavacode.config.CacheConfig    :  Inside cacheManager()...
2018-02-28 20:31:45.985  INFO 9856 --- [           main] s.w.s.m.m.a.RequestMappingHandlerAdapter : Looking for @ControllerAdvice: org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@30946e09: startup dat
e [Wed Feb 28 20:31:37 CST 2018]; root of context hierarchy
2018-02-28 20:31:46.214  INFO 9856 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/cachedemo/v1/companies/{id}/],methods=[GET],produces=[application/json]}" onto public com.betterjavacode.models.Company com.bette
rjavacode.resources.CompanyController.getCompany(int)
2018-02-28 20:31:46.217  INFO 9856 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/cachedemo/v1/companies],methods=[GET],produces=[application/json]}" onto public java.util.List<com.betterjavacode.models.Company>
 com.betterjavacode.resources.CompanyController.getAllCompanies()
2018-02-28 20:31:46.222  INFO 9856 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error]}"onto public org.springframework.http.ResponseEntity<java.util.Map<java.lang.String, java.lang.Object>> org.springframewo
rk.boot.autoconfigure.web.BasicErrorController.error(javax.servlet.http.HttpServletRequest)
2018-02-28 20:31:46.223  INFO 9856 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error],produces=[text/html]}" onto public org.springframework.web.servlet.ModelAndView org.springframework.boot.autoconfigure.web
.BasicErrorController.errorHtml(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse)
2018-02-28 20:31:46.300  INFO 9856 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/webjars/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2018-02-28 20:31:46.301  INFO 9856 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2018-02-28 20:31:46.377  INFO 9856 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**/favicon.ico] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]
2018-02-28 20:31:47.071  INFO 9856 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
2018-02-28 20:31:47.184  INFO 9856 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8080 (http)
2018-02-28 20:31:47.195  INFO 9856 --- [           main] c.b.S.SpringAppCacheApplication          : Started SpringAppCacheApplication in 10.626 seconds (JVM running for 11.552)

In this console output, we will see our log statements Inside redisConnectionFactory, Inside redisTemplate, Inside cacheManager.

Conclusion

In this short post, we showed how to use RedisCacheManager to configure Redis for a spring boot application.

References

 

Caching: How to use Redis Caching with Spring Boot

In this introductory post, we will show how to use Redis caching in a simple spring boot application. In subsequent posts, we will evaluate different factors of Redis caching. But for now, we will try to focus on the simple problem of providing caching to a rest service that provides companies-related data to the user interface. This data is in a database, but caching will help us improve the performance.

 

What you need

  • Java 8
  • MySQL Database
  • IntelliJ Editor
  • Gradle
  • Redis Server and Redis Desktop Manager

Spring Boot Based Rest Service

As part of this post, we will build a simple spring-boot based rest service. This rest service will provide data related to companies which will be stored in mysql database.

We will be using Gradle to build our dependencies in this project. Important dependencies for this project are spring-boot-starter, spring-boot-jpa and spring-boot-starter-data-redis With all the needed Gradle dependencies, our Gradle script will look like below:

buildscript {
  ext {
    springBootVersion = '1.5.10.RELEASE'
  }
  repositories {
    mavenCentral()
  }
  dependencies {
    classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
  }
}

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'org.springframework.boot'

group = 'com.betterjavacode'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = 1.8

repositories {
  mavenCentral()
}

jar {
    manifest {
        attributes 'Main-Class':'com.betterjavacode.SpringAppCache.SpringAppCacheApplication'
    }
    baseName= 'SpringAppCache'
    version='0.0.1-SNAPSHOT'
}

dependencies {
  compile('org.springframework.boot:spring-boot-starter')
  compile('org.springframework.data:spring-data-jpa')
  compile('org.springframework.boot:spring-boot-starter-data-redis')
  compile('org.springframework.boot:spring-boot-starter-web')
        compile('org.hibernate.javax.persistence:hibernate-jpa-2.1-api:1.0.0.Final')
        compile('mysql:mysql-connector-java:5.1.6')
        compile('org.hibernate:hibernate-core:5.2.13.Final')   
        compile('org.aspectj:aspectjweaver:1.8.13')
  testCompile('org.springframework.boot:spring-boot-starter-test')
}

Let’s build a model class for the object Company which will look like below:

package com.betterjavacode.models;

import javax.persistence.*;
import java.io.Serializable;

@Entity(name="Company")
@Table(name="company")
public class Company implements Serializable
{
    public Company()
    {

    }

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private int id;
    @Column(nullable=false)
    private String name;
    @Column(nullable=false)
    private String type;

    public Company(int id, String name, String type)
    {
        this.id = id;
        this.type = type;
        this.name = name;
    }

    public int getId()
    {
        return id;
    }

    public void setId(int id)
    {
        this.id = id;
    }

    public String getName()
    {
        return name;
    }

    public void setName(String name)
    {
        this.name = name;
    }

    public String getType()
    {
        return type;
    }

    public void setType(String type)
    {
        this.type = type;
    }
}

We will not be showing any of the middle layer code which is mostly how the data is going to be built.

Our RestController will use an autowired CompanyManager to fetch company data from the database.

Before we build RestController, we will show the configuration that we have annotated in SpringAppCacheApplication main class.

package com.betterjavacode.SpringAppCache;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;

@SpringBootApplication
@EnableAutoConfiguration
@ComponentScan(basePackages = "com.betterjavacode")
@EnableJpaRepositories(basePackages = "com.betterjavacode.repositories")
@EnableCaching
public class SpringAppCacheApplication
{
  public static void main(String[] args) {
    SpringApplication.run(SpringAppCacheApplication.class, args);
  }
}

Here you can see, we have enabled caching with annotation @EnableCaching.

Now in our RestController class CompanyController , this will show annotation of @Cachable that helps decide when to cache data for the incoming request. This annotation caches data that has been fetched for the request based on configuration.

package com.betterjavacode.resources;

import java.util.List;

import com.betterjavacode.interfaces.CompanyManager;
import com.betterjavacode.models.Company;
import org.hibernate.annotations.Cache;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.*;


import javax.websocket.server.PathParam;

@RestController
@RequestMapping(value="/cachedemo/v1")
public class CompanyController
{


    @Autowired
    public CompanyManager companyManager;


    @RequestMapping(value = "/companies", method= RequestMethod.GET,
    produces = {"application/json"})
    @ResponseStatus(HttpStatus.OK)
    @ResponseBody
    @Cacheable("companies")
    public List<Company> getAllCompanies()
    {
        return companyManager.getAllCompanies();
    }


    @RequestMapping(value = "/companies/{id}/", method = RequestMethod.GET, produces = {"application/json"})
    @ResponseStatus(HttpStatus.OK)
    @ResponseBody
    @Cacheable(value = "company", key = "#id")
    public Company getCompany(@PathVariable("id") int id)
    {
        return companyManager.getCompany(id);
    }
}

Here is a controller, if you see we are caching the data coming from the database with annotation @Cacheable

To make sure data gets cached with Redis server, we will need certain properties where these annotations will help us to cache the data. The properties to configure Redis server are below:

#########################################################################################
## REDIS CACHE
#########################################################################################
spring.cache.type = redis
spring.redis.host = 127.0.0.1
spring.redis.port = 6379

Once you build the project and run it, we will be able to perform the REST requests to fetch data. If we perform the same requests multiple times, we will be able to see the data in redis.

Conclusion

In this post, we showed how to use redis-caching to cache the data for a spring boot based REST service. The code from this post is available to download github

 

Design of REST API for web application

One reason I like to build an application in public is that it keeps me accountable. I can’t run away. If I don’t finish something, it’s ok. At least, I will have something done to show to people. Building in public is not a new idea, a lot of people have used it. In this post, I discuss the design of REST API for Social KPI.

In the previous post here, we discussed the architecture of the application we are building.  This will be an ongoing process as we continue to build our application and evolve.

We will follow the following tips to design REST APIs

  1. We will use Resource to represent object for REST APIs
  2. API endpoint will represent a resource object in the plural. Example – companies, users
  3. We will use HTTP status codes for success or failure of the request
  4. We will use JSON object to represent a response
  5. And we will use versioning to represent a version of APIs

As discussed in the previous post application idea, we will have APIs for companies, users of those companies, customers, clicks, engagements data. While concluding this short post, I want to say that the next post will include database design as well as URL design for REST APIs.

We will be using Spring Boot to build REST API.

In conclusion, I discussed the design of the REST API for the web application Social KPI. If you want to follow the progress, subscribe here.

Architecture of the web application

In my last post design, I discussed the idea that we are going to work on building a web application. I detailed the user flow, but I missed out on some points about security and session management. I will add the details of architecture of social KPI web application.

Name of the application

Before we discuss the application, we still haven’t decided on the name for the application. This web application will indicate the performance of a small business in social media. Basically, this is a free tool for marketing and depending on how small businesses use social media, they will be able to build a campaign for their business. If small businesses are not using social media, they are already at a disadvantage. This is just a pie in the big social world. That brings me to the purpose of the application to provide social key performance indicators (social KPIs) to businesses. So the name of the application will be SocialPie.

Security and Session Management

We will use Spring Boot. We will be using spring security elements to build authentication and authorization aspect of the application. I will definitely include the details of this component when we will start working on building the application. In a previous post spring security, I have discussed how to use spring security for authentication.

For managing a session, we will be using spring provided service based on Redis. We will also be using caching considering we will be connecting to Facebook, Twitter, and Instagram APIs, so we can keep the data in cache for pre-decided time. This will be beneficial from a performance perspective. We will be using Redis caching with our own cache manager to handle caching.

I will try to include all these elements in the architecture diagram that we will be building in this post.

Architecture

Architecture of web application Social KPI

 

Conclusion

In this post, we created an architecture for our web application Social KPI. In the next post, I will detail another user flow with some class diagrams and explain each service in detail. The application will be based on microservice architecture.