Thursday, March 27, 2014

Cassandra running out of memory heap space

I have configured Cassandra 2.0.4 in my laptop which was working fine for few weeks.  My machine configuration is below -







On a day-2-day basis, I have been truncating the database and recreating frequently.  After few weeks, when I tried to start Cassandra, I started getting the below error - 

java.lang.OutOfMemoryError: Java heap space
at org.apache.cassandra.io.compress.CompressedRandomAccessReader.<init>(CompressedRandomAccessReader.java:64)
at org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:44)
at org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:949)
at org.apache.cassandra.io.sstable.SSTableScanner.<init>(SSTableScanner.java:72)
at org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:796)
at org.apache.cassandra.db.RowIteratorFactory.getIterator(RowIteratorFactory.java:88)
at org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1329)
at org.apache.cassandra.service.StorageProxy.getRangeSlice(StorageProxy.java:806)
at org.apache.cassandra.thrift.CassandraServer.get_range_slices(CassandraServer.java:697)
at org.apache.cassandra.thrift.Cassandra$Processor$get_range_slices.process(Cassandra.java:3202)
at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889)
at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:187)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)


I have made the below changes to Cassandra.bat and the problem got resolved.

























I have updated the two highlighted values from -Xms1G^, -Xmx1G^ to -Xms2G^, -Xmx2G^ by allocating 2GB of heap memory for Cassandra. 

How to connect to Cassandra server running in Amazon EC2 using Cassandra client in local environment?

Below steps need to be followed, if you want to connect to Cassandra instance running in Amazon EC2 using the Cassandra client running in local environment.

  1. Goto EC2 Management Console
  2. Click on Security Groups under Network and Security on the left menu
  3. Select your Security Group associated with your instance and click on Inbound Tab
  4. Find out what is the source for ALL Port (Service). If the source is sg-xxxxxxxx, then delete the same.
  5. Add a new rule by selecting the below options: Create a new rule: All Traffic Source: 0.0.0.0/0
  6. Add Rule and click on Apply Rule changes
  7. To verify if you are able to connect to Cassandra server, run the cassandra client from your local machine like below:
cassandra-cli -host ec2-XX-XXX-XX-XXX.us-west-2.compute.amazonaws.com -port 9160

How to release the Cassandra port 7199 used by other application?

After downloading the Cassandra database from Apache Cassandra website and when you try to execute the cassandra.bat under /bin folder, you may get the below error if the port 7199 is used by some other application:

Starting Cassandra Server Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 7199; nested exception is: java.net.BindException: Address already in use: JVM_Bind. Can anyone assists me in running cassandra successfully.

How to identify which application is blocking port #7199 and how to kill that?


Open the TCPView application and sort the output by Port
Click on the record which points to Port - 7199
Right click and "End Process"
Now, run the Cassandra.bat and it should work.