Microservice Monitoring

In microservice world there are many independent isolated services runs in parallel and there are always concerns that which instance/s of service is running fine or which are not. In monolithic world it was quire simple. For small to moderate traffic there will be 3/4 instance sufficient to handle the load and each instance can be easily monitored through APM, access logs and even some time through instance level monitoring (As one VM will only have one Application instance).

In case of microservice world the complexity increases N fold as each service we are starting embedded tomcat and each service is having multiple API calls which are either call directly by client or by other microservice.

In Microservice architecture principle each service should be isolated and independent, same principle holds true for even monitoring.  Each service should be self monitored and should take part in overall scaling, coordination and orchestration.

For any human BP, Sugar, Pulse etc are the metrics to determine whether that person is healthy or not in same way there are various metrics each microservice should expose to determine the health of that particular service.

For monitoring particular instance which are running bunch of microservice following metrics are critically important:

1. CPU Usage
2. Memory Usage and Swap
3. Bytes transferred across the network.
4. Disk read write speed.

For monitoring particular microservice following metrics become critically important:

1. CPU usage by the container.
2. Free and used memory statistics in therms of heap memory.
3. Garbage Collection performance.
4. Class Loaded and threads
5. API Response time.

All above statistics can be easily monitored, alerted and visualised using the following tools:

1. cAdvisor -- Monitoring the Docker Container and System.
2. Spring Actuator  -  Getting application performance metrics. 
3. Prometheus -- Time Series DB 
4. Grafana --  Graphs and Monitoring Dashboards.

Let's start making all things to work together. For better understanding lets take use case:

Admin looks at the dashboard for monitoring the single microservice which is deployed as docker container.

Spring Boot:

Lets start real work create the some spring boot application and add the following dependency:

  	<dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-actuator</artifactId>
         </dependency>
 	<dependency>
 		<groupId>io.prometheus</groupId>
 		<artifactId>simpleclient_spring_boot</artifactId>
 		<version>0.1.0</version>
 	</dependency>
 	<dependency>
 		<groupId>io.prometheus</groupId>
 		<artifactId>simpleclient</artifactId>
 		<version>0.1.0</version>
 	</dependency>
 	<dependency>
 		<groupId>io.prometheus</groupId>
 		<artifactId>simpleclient_hotspot</artifactId>
 		<version>0.1.0</version>
 	</dependency>
 	<dependency>
 		<groupId>io.prometheus</groupId>
 		<artifactId>simpleclient_servlet</artifactId>
 		<version>0.1.0</version>
 	</dependency>

Create Configuration file and add the following:

 @Bean
    @ConditionalOnMissingBean
    CollectorRegistry metricRegistry() {
        return CollectorRegistry.defaultRegistry;
    }

    @Bean
    ServletRegistrationBean registerPrometheusExporterServlet(CollectorRegistry metricRegistry) {
        return new ServletRegistrationBean(new MetricsServlet(metricRegistry), "/prometheus");
    }

    @Bean
    public SpringBootMetricsCollector metricsCollector(Collection<PublicMetrics> publicMetrics) {

        List<Collector> collectors = new ArrayList<>();
        collectors.add(new StandardExports());
        collectors.add(new MemoryPoolsExports());
        addCollectors(collectors);
        SpringBootMetricsCollector springBootMetricsCollector = new SpringBootMetricsCollector(
                publicMetrics);
        springBootMetricsCollector.register();
        return springBootMetricsCollector;
    }

Now needs to create sample api:

@RequestMapping(value = "/postjson", method = RequestMethod.POST)
    public String postjson(@RequestBody String json) throws IOException {

        System.out.println("Request: "+json);
        try {
            Random ran = new Random();
            int x = ran.nextInt(6) + 5;
            TimeUnit.SECONDS.sleep(x);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        return "Received:"+json;
    }

Above API is very simple one upon each call it will wait from 5 - 10 seconds and respond back.
This will enable us to see up and down in monitoring graphs.

Once you run the above application you will able to see the metrics in prometheus format in following url:
http://localhost:9099/prometheus

Out put will look like:

# HELP httpsessions_max httpsessions_max
# TYPE httpsessions_max gauge
httpsessions_max -1.0
# HELP httpsessions_active httpsessions_active
# TYPE httpsessions_active gauge
httpsessions_active 0.0
# HELP mem mem
# TYPE mem gauge
mem 362111.0
# HELP mem_free mem_free
# TYPE mem_free gauge
mem_free 125518.0
# HELP processors processors
# TYPE processors gauge
processors 8.0
# HELP instance_uptime instance_uptime
# TYPE instance_uptime gauge
instance_uptime 412918.0
# HELP uptime uptime
# TYPE uptime gauge
uptime 415762.0
# HELP systemload_average systemload_average
# TYPE systemload_average gauge
systemload_average 4.4248046875
# HELP heap_committed heap_committed
# TYPE heap_committed gauge
heap_committed 313344.0
# HELP heap_init heap_init
# TYPE heap_init gauge
heap_init 262144.0

...

# HELP gauge_response_postjson gauge_response_postjson
# TYPE gauge_response_postjson gauge
gauge_response_postjson 10032.0

Here we had created our custom metrics gauge_response_postjson which tells us the last response time of postjson API. As we are refreshing every five seconds we will get last response time for every five seconds which we will plot as a graph.



Prometheus:

Lets starts configuring the prometheus we will use the docker container for the same.

Configure the prometheus for our end point:
  - job_name: 'finx_service'
    metrics_path: /prometheus
    scrape_interval: 5s
    static_configs:
      - targets: ['192.168.1.178:9099']

Start prometheus : 
docker run -d -p 9090:9090 \
              -v $PWD/prometheus.yml:/etc/prometheus/prometheus.yml \
              prom/prometheus
You should able to access the prometheus using the following url:
http://localhost:9090/
Make sure that targets are working (state=up) fine http://localhost:9090/targets

Grafana:
Start Grafana using docker: docker run -d --name=grafana -p 3000:3000 grafana/grafana
Login into http://localhost:3000 as admin/admin.
In order to stimulate the load we will fire the following command using curl multiple times.

curl -H "Content-Type: application/json" -X POST -d '{"username":"xyz","password":"xyz"}' http://localhost:9099/postjson

Now configure the Grafana dashboard shown below:

System Monitoring:

Now let's monitor the Docker Container and System Statistics using the cAdvisor.

Start the cAdvisor as follow:

sudo docker run \
--volume=/var/run:/var/run:rw \
--volume=/sys:/sys:ro \
--volume=/var/lib/docker/:/var/lib/docker:ro \
--publish=8080:8080 \
--detach=true \
--name=cadvisor \
google/cadvisor:latest

Check the following url whether metrics are getting generated:

http://localhost:8080/metrics

Note use might wondering cAdvisor has its own dashboard at http://localhost:8080/ then why we are moving its metrics from cAdvisor to Prometheus and plotting graph in Grafana. Reason is that cAdvisor only keep last 15 min of data so if you want to see for more than 15 min then the data is overridden so we send the metrics to prometheus and use Grafana to visualise it.

You should able to see following targets in Prometheus:

Configure the Grafana dashboard by using free Docker and System dashboard from  Docker Dashboard

Above source code is available at GitHub.




Next blog I will cover how to do monitoring orchestration and auto discovery.

Send Message To RabbitMQ in Transaction

What do I meant by sending message in Transaction ?

Sending message in transaction mode mean if the transaction is succeed then message/event should be published. But in case of transaction failure or rollbacks the message/event shouldn't be published.

In many Enterprise Application transaction is the bread and butter of the architecture. Traditional architecture is highly dependent on transaction. Even in new era of Microservices  there comes the need to send the messages across the components in some what transaction way to maintain the data integrity across the component.

Let's take some simple example of microservice: 

Here there is simple use case once the order is placed it will notify the user about the order confirmation. In microservice architecture, order and notification are two different component having their own JVM and DB. Definitely this flow should be async and also consistent. 

Here there is a need/requirement  that notification should not trigger if the order placement fails. Notification should only be delivered if the order transaction succeed. So there is need to send message to RabbitMQ  ( Any Broker) in transaction mode.

Currently RabbitMQ  doesn't participate/support the transaction there is way for two phase commit, but there are pitfalls which I am not going to discuss here. 

So question comes how we can leverage the RabbitMQ power to use it under transaction answer is Orchestration. 

Based on my experience there are following options each having their pos and cons.

Option -1 Orchestration Using RabbitMQ Itself. 

For this option create some broker framework which can get embedded into the application or it can run standalone.  

1. Here application create the payload and generate unique hash value.
2. Under transaction it send message to temp queue and writes the hash value to some table in DB.
3. Broker Framework listen to the temp queue and upon receive of any message it checks for the hash value stored in step-2.
4. If the hash value is present it forward the message to original queue and if not present it pushes back to the temp queue.
5. After no of retry the message will get discarded / moved to error queue reason being transaction might not succeed and hash value is never written to DB. 

Challenges:

• If the transaction takes longer the broker will receive the message before the transaction ends and due to which retry might get triggered more frequently.
• If volume is high as in many case message queued back after retry will take longer time to reach destination.

+Ve:

• Highly scalable. In case of embedded it autoscale as the node scale.
• Easy to monitor and maintain.
• Event Driven

-Ve:

• Data Cleanup
• More delay in case of longer transaction time

Option -2 Orchestration Using Poller.

This is very traditional approach where poller will poll the transaction data and publish the same to queue.

1. App stores the payload data in transaction. ( DB Isolation Level read committed).
2. Poller polls the data and push it to queue.
3. Once the data is push the data is cleared from DB via update / delete.

Challenges:

• Poller scaling.
• Heavy read and write operation in DB.
• Poller memory issues (How much data to poll ).

+Ve:

• Truly transaction.

-Ve:

• Difficult to scale.
• High Maintenance and tuning requirements. 

Option -3 Orchestration In Memory.

Here the message will be stored in memory and will delivered to MQ after transaction succeed.

1. Application will store the message in memory till the transaction completes.
2. Once the transaction is completes in memory message is pushed to queue.
3. Shutdown hook is implemented which will write the in memory data to disk if shutdown is triggered so that once the application comes up it will able to check the transaction and push to queue.

Challenges:

• Transaction complete triggering.
• Memory management to some extent.
• Shutdown and Restart.

+Ve:

• Very easy to implement ( Remember to take concurrency effect).
• Scalable.

-Ve:

• Rollback cleanups
• High maintenance in multi node scenario.

Option -4 Orchestration via coordinator.

Here the message will be send to rest broker which can act like coordinator which will coordinate for the transaction processing.

1. Application submit the payload to the broker which return the unique token back in response.
2. On post processing of the transaction the application submit the token back to the broker, and broker send the message to queue.
3. Broker work on time based it will hold the token for stipulated time and will discard the message if the token is not received in stipulated time.
4. Optionally if broker needs to scale then broker might store the message temporarily in some distributed cache like redis etc.

Challenges:

• Transaction completing triggering. ( What if the node goes down after the transaction is complete and was not able to submit the token).
• Complex development and monitoring.

+Ve:

• Scalable

-Ve:

• High Monitoring and fallback.

Conclusion:

Which option to choose depends on the requirement.  Example Option-3 is suitable for low volumes and option-1 and option-4 can be used for high volume loads. Option-2 which is pure transactional stuff, but not event driven and difficult to scale needs rethinking before adopting it.

If you believe in KISS (Keep it simple Stupid) principle then go for Option-1. As per me this is easiest to implement, fault tolerance and highly scalable

Before starting, evaluate various options via POC and find out the best which suits you. 

Happy coding.

JUNIT - Method Contract Testing

What is method contract testing:

Form the developer stand point it is very important to write Junit. I my career I had seen many different method/process/ways people write junit. Some write serious test cases, some writes for just sake of writing the test case and some test cases even puzzled me what exactly test case want to accomplish. 

Now as everyone is moving towards continuous integration (CI), automated builds, self-testing and rapid development writing good junit become the most critical part of development cycle.

In any development the most important aspect to test is Object, Method and its behavior with respect to change in its/dependent object state and evaluation path along with non functional aspects.

Any good JUNIT should revolve along the first two, change in state and evaluation path where as non functional aspects should be covered more precisely in integration testing.

While doing JUNIT developer needs to concentrate to method under test and all input / dependencies needs to be mocked. Test exactly how method will react to input, change in object state. Doing so if each and every method has good test case then eventually you will end up in writing good test case which can cover almost all scenarios ( That depends upon how good test case is written).

Coming to the point each method under test has certain method contracts. Following are different types of method contract:

1) Input Contract.

2) Output Contract.

3) Class Contract.

3) Exception Contract.

4) Behavioral Contract.

5) Non functional Contract.

Input Contract:

1) Null check - Method may or may not check null validity for input parameters.

In this case either current method or calling method should have null check unit test case.

2) Input Modification - Method may or may not modified the input objects.

In this case proper assert needs to be there for input object passed to check for modifications.

Output Contract:

1) return value contract - Based on the signature method can return new object or existing one.

Here proper asserts needs to be there to check for new object or existing object modifications.

2) return by reference - Method can return by modifying the input object which is passed by reference.

Proper assert needs to be there to unit test the same.

3) void contract - Method doesn't return any value.

Assert all the input / class object for any modification.

Class Contract:

1) Class contract - Method can change / initialize the accessible class /static variables.

Provide proper assert to track this changes.

Exception Contract:

1) forwarded exception - Method forward the exception instead of catching.

Mock the exception and assert the same.

2) new exception - Method creates and throws new exception.

Assert the new exception whether it is thrown or not.

Behavioral Contract:

Method can have one or many behavioral contract.

For all behavioral contract proper unit test and assert needs to be there.

Example:

If null is passed as input I will return null object which caller needs to handle.

If dependent call throw exception I will not rollback the changed input /class object.

Non functional Contract:

Method can have various non functional aspect like logging , transaction, concurrency etc.

All non functional aspect should be cover as part of integration testing and it should not be included as part of unit testing.

Heap Dump Basics

Taking and using the heap dump information  is very critical for successful career in Java Development. It will be very hard to believe that if you had done successful Java development, you had never taken or analyzed heap dump.  In my career  I had done many production deployments and  solved many critical production and non production issues where I think without heap dump it will be very tough and some time almost next to impossible to solve certain issues. I had be approached by many how to take and analyse heap dump especially who are new or not mature to Java development. So this blog might help them to get some basic to some detail information about the heap dump in step by step order. Lets start with exactly what is heap dump ?.

What is Heap Dump:

Heap dump is a snapshot of the memory of a Java process at a certain point of time. It contains the information about objects loaded in memory, classes,

garbage collection roots, thread stack, local variables.

Using this information any body can get the following details:

1) How many classes loaded.

2) How many objects created.

3) How many heap space available and occupied.

4) What were the JVM parameters/options used for that run.

5) Heap used by each object type.

6) Thread status.

7) Thread call stack - exactly which method was getting evoked during that time.

8) Unreachable objects and many more. But this information is at the point of heap dump and it doesn't reflect the JVM past state.

When to take Heap Dump and why :

1) When your application seems hanging.
2) When there is sudden spike in memory.
3) When application crash due to out of memory issue.
4) When you want introspection into JVM at particular time or event.

Triggering of Heap Dump:

1) Triggering Heap Dump under live running application.
2) Triggering Heap Dump when the application goes out of memory.

Note: When you trigger the heap dump all the active threads are stopped (duration depends on the heap size) . This can lead to performance issue as many critical application cannot tolerate such activity.

Ways to Trigger Heap Dump:

Heap Dump on out of memory: 

java  -XX:+HeapDumpOnOutOfMemoryError  -XX:HeapDumpPath=   ....

or add to JAVA_OPTS if you are using tomcat etc.

Manually taking heap dump of running application:

Find the process id of the application:

jps -lv or ps aux | grep java.  -- Recommended is jps 

then take heap dump with jmap
jmap -dump:file=<> <>
jmap -dump:file=heap_dump.hprof 4208

Manually taking heap dump using the UI tool:

Start jvisualvm --> Navigate to the process --> go to monitor tab --> Click Heap Dump Button.

Knowing the object size and no of instances:

jmap -histo:live <>
jmap -histo:live 4208


Note you can also take the heap dump by enabling the JMX.

In next blog I will give the tips to analyze the Heap Dump

MongoDB Essential Facts

Transactional Support:

1) MongoDB provide ACID in following ways:

• Atomicity - MongoDB provides only a document-wide transaction:  Writes are never partially applied to an inserted or updated document.  The operation is atomic in the sense that it either fails or succeeds, for the document in its entirety.
• Consistency – Consistency is achieved by replica set.
• Isolation – DB level locking from version 2.2 onwards
• Durability – JournalCommitInterval configuration options.

2) You need to model the data correctly to achieve the atomic operation.  Example

3) Even if  in future if there is requirement for having atomicity between multiple collections that can be achieve via two phase commit.

4) Ideally all atomic MongoDB operation should be performed from application layer or by scripting.

JOINs:

1) 1. JOIN’s requirement can be met in following ways in MongoDB:

•  DBRefs - Referencing one document to other via _ids.
• Embedding - Putting the data into single document. .

2) When to Embed and when to use DBRefs:

• If the document is huge and is refereed in many document then it is prefer to use DBRef. Doing so if document needs updating it is done once rather than many times. Also it saves the disk space and performance is better as lesser I/O ( Due to document caching).
• But if there is one-one relationship and is not refereed to other document then it is always better to embed, irrespective to size of document. If updates are zero and relationship graph is small then also it is advisable to embed the document.
• Again it depends on the user requirement also whether there is lazy loading required, whether data is write once and read many times, read/write ratio, atomic data requirement, ACID requirement and many other factors which I leave to user to decide.

Database level locking:

 For ACID locking is much more important then any other aspect.  Previous version of MongoDB there was global read/write lock which is slow in performance but now from new release 2.2 things had improved to great extent and hope future release will also add more value.

1) From MongoDB version 2.2 there are the improvements in locking:

•  Elimination of the global reader/writer lock.
•  Locking at database level.
•  PageFaultException architecture – yield lock on page fault.

2) PageFaultException architecture improves concurrency within a single collection,which meets most of normal requirements.

3) DB performance depends on the read/write ratio, so it is recommended to do benchmark before taking any decision.

4) In future MonogoDB is going to have document level locking (latching).

File Sizes :

File size matters a lot from maintenance and from performance perspective. 

Please refer the MongoDB documentation for in depth details.

1) Each BSON document has 16 MB limit in MongoDB, if your document span across 16 MB then you need to use GridFS.

2) Following are the various options to shrink the file if required:

• db.repairDatabase(). 
• Compacting particular document: db.runCommand({ compact : 'User' }); 

3) Test of file size:

• No of collection:2
• No of Documents:
• Collection 1: 216147
• Collection 2: 245712
• Total No of Keys in each collection: 30
• Data type used: integer, date, string, double.
• Average length of each key : 10 characters.
• Total physical file size for that db: 256 MB (268,435,456 bytes) on Windows platform.

Above  figures are done in local isolated environment and it is better to do benchmark testing on your environment and based on your data requirement.

Security:

1) MongoDB needs to be deployed in trusted environment.

2) MongoDB provides basic support for authentication with the auth setting, which is disable by default. 

3) MongoDB can have control access in trusted environment.

4) In production MongoDB will not be installed in an environment which should not be directly accessible via web.

When to use Mongo Db :

1) When your data is not much normalized.
2) When you want to store unstructured data.
3) When you want data flexibility.
4) When you are not much concern about the ACID properties, ideal case most reads operations.
5) If you want to dump data dynamically without even thinking of the schema, ideal case dump as it is form.
6) There is no need of complex transactions.
7) If you want to achieve high scalability, but considering the above aspects.
8) If your requirement is to support high volume traffic.

It is always advisable to do performance/benchmark testing to meet your requirement. 

Production Deployments

File Copy Using Java -- Basic and Fastest Way

Recently I had faced with situation to write simple copy command in Java, so I written the same using the plain old style and give it back to developer to implement and unit test the same. To my surprise developer comes back and reported me that my copy code is performing badly. Reason he want to use my copy code to copy at least 20 GB of file data.

So now I was faced with challenge to implement the fastest copy code. The first thing came to my mind was to use the nio API of java.

Copy using the java mainly depends on the following factors which you can control:

1) Buffer Size
2) Reading the data in efficient manner.

Buffer Size:

Buffer size is mainly dependent on operating system block size and CPU Cache. Example windows default block size is 4K (4096 Bytes).

Now if some one configure the block size of say 4500 bytes then two blocks (2* 4096) will be read and put it into the memory. So it means you had read extra bytes which will not be used and you pay the price for disk to ram read. Many times OS cache the blocks you read so next read it will be used but again here you pay the price of RAM and cost of RAM to CPU Cache copy. So ideally the buffer size should be equal to the block size or multiple of it ,so that every time you read it will be full block size read and there is no wastage. Remember cost of RAM to CPU Cache is much lesser than the cost of disk to RAM reads.

Again it depends on the CPU how it moves the data between L3 to L2 cache and complexity increases. So if you run your program under different cache size it gives different result. As per my experience the best buffer size is 8192 bytes even the java default buffer size is 8192. I will suggest to try
different buffer size and select the optimal one which best suites your environment.

Data Reading:

Most inefficient way of reading the text data is

while ((line = in.readLine()) != null) {
...
}

Here you are creating the String object every time you read a line. Lets say you are reading 10000 lines of file that means you are creating 10000 string objects and unnecessary time is spend in object creation, allocation and garbage collection. Instead of String you can use the character array.

For Binary Copy many write the code as below:
BufferedInputStream in = new BufferedInputStream(fis);
BufferedOutputStream out = new BufferedOutputStream(ost);
FileInputStream fis = new FileInputStream(src);
FileOutputStream ost = new FileOutputStream(dst);
byte[] buffer = new byte[8192];

while (true) {
int amountRead = in.read(buffer);
if (amountRead == -1) {
break;
}
out.write(buffer, 0, amountRead);
}

If you observer the out.write method the buffer and amount read are same so every time it will be flushed. So if file is huge it will have significant impact. So here while creating the new BufferedOutputStream(ost, size) use the size option and also use the write method instead of the previous one.

After the nio package is releases there are various option to read out of which one of very interesting is the ByteBuffer class.

ByteBuffer read the data repeatedly it keeps the track of the last byte written so that you can be assured of keep writing and rest it will take care. It doesn't mean that you can read endless data ByteBuffer has the limit. Once the limit is reached you need to flip.

ByteBuffer had following concept:

* Capacity - Size of the internal byte[].
* Position - Last byte index filled.
* Limit - Limit = capacity while reading and one past last filled byte while emptying.
* Mark - Bookmark ( optional )

ByteBuffer data is read from buffer starting position to read and while writing data is written from starting at ByteBuffer position upto limit. If you observer the position will advance till the limit and no further reading will be possible as position == limit. So you needs to do flip in ByteBuffer which will basically sets the position to 0 (start), and limit to the position
(previous position to be exact, or rather the end of useful input).


Different ways of Copy:

1) Conventional Way Of File Copy:

public void copyFile(File src, File dst, int buffLen) throws IOException {

BufferedInputStream in = null;
BufferedOutputStream out = null;
FileInputStream fis = null;
FileOutputStream ost = null;
byte[] buffer = new byte[buffLen];
try {
fis =
ost = new FileOutputStream(dst);
in = new BufferedInputStream(fis);
out = new BufferedOutputStream(ost);
while (true) {
int amountRead = in.read(buffer);
if (amountRead == -1) {
break;
}
out.write(buffer, 0, amountRead);
}
out.flush();
} catch (Exception e) {
e.printStackTrace();
} finally {
if (in != null) {
in.close();
}
if (out != null) {
out.close();
}
if (fis != null) {
fis.close();
}
if (ost != null) {
ost.close();
}

}
}

Here we are using the Buffered Stream to read and write.

2) Using the nio package and lets JVM decide the best way to copy:

public void copyUsingFileChannelIntenalTransfer(File source, File destination) throws IOException {
FileChannel in = null;
FileChannel out = null;
try {
in = new FileInputStream(source).getChannel();
out = new FileOutputStream(destination).getChannel();
// JavaVM does its best to do this as native I/O operations.
in.transferTo(0, in.size(), out);
// Closing file channels will close corresponding stream objects as well.
} finally {
out.close();
in.close();
}
}
Here we are depending on the JVM to decide the best buffer to make it as native I/O operation.

3) Copy using ByteChannel and Bytebuffer :
public void copyUsingByteChannel(File source, File destination, int bufferSize) throws IOException {
final InputStream input = new FileInputStream(source);
final OutputStream output = new FileOutputStream(destination);
final ReadableByteChannel inputChannel = Channels.newChannel(input);
final WritableByteChannel outputChannel = Channels.newChannel(output);
try {
final ByteBuffer byteBuffer = ByteBuffer.allocateDirect(bufferSize);
while (inputChannel.read(byteBuffer) != -1) {
// preparing for write by moving the limit to position
// and making position to zero
byteBuffer.flip();
// Writing to channel output stream.
outputChannel.write(byteBuffer);
// If there is any bytes left between postion and limit transfer
// it to front else same result as clear.
byteBuffer.compact();
}
//There are chances that after compact some bytes might left between
// position and limit so that also needs to be written.
byteBuffer.flip();
// Writing the remaining bytes if there is any
while (byteBuffer.hasRemaining()) {
outputChannel.write(byteBuffer);
}
} finally {
inputChannel.close();
outputChannel.close();
}
}
Here we are using the ByteChannel and ByteBuffer to copy please take note of comments for understanding.

4) Copy using MappedByteBuffer :

public void copyUsingMappedByteBuffer(File source, File destination) throws IOException {
FileInputStream fi = new FileInputStream(source);
FileChannel fic = fi.getChannel();
// Here also we are relying on the jvm and os to determine the
// buffer size
MappedByteBuffer mbuf = fic.map(
FileChannel.MapMode.READ_ONLY, 0, source.length());
fic.close();
fi.close();
FileOutputStream fo = new FileOutputStream(destination);
FileChannel foc = fo.getChannel();
foc.write(mbuf);
foc.close();
fo.close();
}

Here we are using the Mapped Byte Buffer and completely relying on jvm to determine the optimal buffer size.

Result:
After rigorous testing with different file size and different buffer size I came to
conclusion that the option no 3 is the fastest and consistent.
We were able to achieve more than 65% of performance improvement by opting to option no 3.
I will update this blog with the throughput data soon.