Posts

Using Logstash for large scale log analysis along with MongoDB

Image
This simple demo will give you an idea on how to use the logstash for large scale analysis of logs along with MongoDB. MongoDB can be used to store the large amount of logs. These logs further can be used to perform analytic by bringing into SQL DB as per requirement or as per time frame. Step 1: Installing the Contrib package which has MongoDB plugin   Option 1:  Using Script    vaibhav@ubuntu:~/logstash-1.4.2$ bin/plugin install contrib   As per ( https://github.com/elasticsearch/logstash/issues/1658 ) if this fails there is option 2 to install contrib package. Option 2: Install the contrib package in same directory, in which the Logstash is installed. wget http://download.elasticsearch.org/logstash/logstash/logstash-contrib-1.4.2.tar.gz  tar zxf logstash-contrib-1.4.2.tar.gz Step 2:  Modify the configuration file for the logstash with with the credentials for the MongoDB as output. Input to the logstash is simp...

Part 2:Bangalore Memories

Part 1: One year@lnt:Chennai Memories 21 st June 2012 we landed in Bangalore, leaving all our batch mates in Chennai. The very first difference I felt when I entered in Bangalore was climate, cool and pleasant, unlike the super hot climate of Chennai. Initial days were somewhat tough to adjust here, as we were missing our buddies back in Chennai. But life moved on everyone over here was assigned to projects, new team, and new friends. Even though we were in same campus, we could hardly see each other once a while, casual hi –byes. I wonder how suddenly things got changed, few days back; we were enjoying together without any formalities. But Its life…move on!!! Project and work As we say These IT companies lacks innovation in work but I must say they do lots of innovation in resource allocation to project. I am from computer science background but don’t know what they saw in me ,and put me in GSM/GPRS telecommunication project.(may be they saw some G’s I will inv...

One year @lnt Infotech:Part 1-Reliving Chennai memories

After long time came back to write something the occasion which urged me to write, is day i.e.24 April.  Exactly one year ago on 24 April 2012 journey of actual life started for me, start of professional career with lnt infotech @Chennai.  Thought of writing something on this occasion  came in my mind to cherish  our  memories and take you all again to relive our old Chennai days ,I am  sure this memories will take you back in to past and u will also feel nostalgic like what I am feeling now.  so here I go…… MUMBAI --------->CHENNAI I was very much excited as, I had never been away from my family this was first time i was going to live my independent life, going away from family and friends. Journey started on 21st April with two persons Krutin Palsodkar and Rohan Mande. I was knowing both of them only from past two days as from facebook , we  got to know that we are going to board same train so it was totally great experience meeting...

या निषेधाचा निषेध ......!!

१३  जुलै २०११ , मुंबई साखळी बॉम्बस्फोटाने  हादरली. एकामागून एक ३ स्फोट दादर, ओपेरा हौस आणि झवेरी बझार. २१ निरपराध लोक मृत्युमुखी पडली तर १०० हून अधिक जखमी . सगळीकडून या स्फोटाचा निषेध व्यक्त करण्यात आला,मग यात social networking sites तरी कशा मागे राहतील.याच वेळी मी होतो facebook वर, सगळे जन status update टाकून  आपले दुख व्यक्त करत होते.सहानुभूतीच्या status updatecha  जणू पूरच  आला होता facebook वर. काळे DP लावून निषेध केला जात होता.२-३ दिवस निषेध अगदी जोरावर आणि मग आठवडाभरात ते काळे DP गायब. ७ सप्टेबर २०११, पुन्हा तीच घटना, जागा मात्र वेगळी. यावेळी निशाण्यावर होती दिल्ली. पुन्हा बरेच निरपराध लोक यात बळी पडली.याच संध्याकाळी मी पुन्हा facebookvar. पण विशेष म्हणजे यावेळी एक status update  नाही  निषेधाची :-(  . कोणी आपले DP  काळे केले नाहीत.                                                          ...