Monday, June 8, 2015

New Approaches to Enhance OS Security

The 5th presentation of NSOM module..

Friday, June 5, 2015

The 4th Summer in Europe..

Finally summer is here in Europe, once more. We had the EMJD-DC spring event in Barcelona. I was in Barcelona 31st May - 4th June. This was my second time to be in Barcelona. It is a beautiful city with lovely beaches along the yellow metro line. Last time it was in 2013 winter for the EMDC winter event. So much of deja vu. As with many other travels, this trip too was mostly spent inside the university, due to the scheduled presentations. However, I had the last day to enjoy the city and its beach.

These days I was able to experience "selective animal right" campaigns against the consumption of dog meat in China. Every day, millions of animals - cows, rabbits, ducks, hens, and cute little baby piglets are tortured in farms for the meat all over the world. Some chose to consume dogs, and everyone has become an animal rights activist. In my opinion, unless you are a vegetarian, you have nothing much to voice here. Majority of the animals that end up in our plate have a cruel life and death in the animal farms (you may have seen a million videos from PETA on this). I don't think dog meat tastes good. Feels disgusting. But it is funny to see the selective animal love. Eat whatever you want, as long as it is not illegal where you live. I don't mind.

I am not a vegetarian. Time to make some chicken curry. The plants in my apartment have started to grow with little flowers. Long sunny and hot days. However, it is going to be another busy summer as usual.

Saturday, May 30, 2015

Configuring Drill with HDFS

Apache Drill can be used to query Hadoop and HDFS in a very efficient way. The steps are as below. Refer to the linked pages for each step.

1. Configure Hadoop 

There may be some errors encountered when executing Hadoop with Java 8 with the default configurations. Further, formatting the file system will fix any error from Hadoop.
  $ $HADOOP_HOME/bin/hdfs namenode -format


2. Start Hadoop NameNode daemon and DataNode daemon -
$ $HADOOP_HOME/sbin/start-dfs.sh
Starting namenodes on [localhost]
localhost: starting namenode, logging to /home/pradeeban/programs/hadoop-2.7.0/logs/hadoop-pradeeban-namenode-llovizna.out
localhost: starting datanode, logging to /home/pradeeban/programs/hadoop-2.7.0/logs/hadoop-pradeeban-datanode-llovizna.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /home/pradeeban/programs/hadoop-2.7.0/logs/hadoop-pradeeban-secondarynamenode-llovizna.out

Once you are done, stop the daemons with

$ $HADOOP_HOME/sbin/stop-dfs.sh

3. Browse the web interface for the name node - http://localhost:50070/

4. Configure Drill

5. Launch Drill in Embedded mode -
$ $DRILL_HOME/bin/drill-embedded 

6. Browse the web interface for Drill - http://localhost:8047/

7. Configure Hive

$ $HADOOP_HOME/bin/hadoop fs -mkdir       /tmp
$ $HADOOP_HOME/bin/hadoop fs -chmod g+w   /tmp
$ $HADOOP_HOME/bin/hadoop fs -mkdir       /user/
$ $HADOOP_HOME/bin/hadoop fs -mkdir       /user/hive
$ $HADOOP_HOME/bin/hadoop fs -mkdir       /user/hive/warehouse
$ $HADOOP_HOME/bin/hadoop fs -chmod g+w   /user/hive/warehouse

Configure Hive Metastore.
Configure with MySQL.

http://sanjivblogs.blogspot.pt/2014/12/install-and-configure-hive.html  

8. Run Hive Metastore and Hive
$ $HIVE_HOME/bin/hive --service metastore &

Start HiveServer2
$HIVE_HOME/bin/hiveserver2

Alternatively, Hive server1 can be started from 
$ $HIVE_HOME/bin/hive

If you encounter URISyntaxException, have a look at this.

9.  Configure and enable Storage Plugin for Hive in Drill
{
  "type": "hive",
  "enabled": true,
  "configProps": {
    "hive.metastore.uris": "thrift://localhost:9083",
    "hive.metastore.sasl.enabled": "false"
  }
}


10. Query Hive from Drill.
SELECT firstname,lastname FROM hive.`customers` 

11. This discusses configuring Drill with Mongo.

Completion of EMDC - II

KTH President at the Welcome Reception
Finally, we had the KTH graduation on the 27th of May. It was pleasant to meet many of my Erasmus Mundus friends in Stockholm after a long time, in a familiar place. Remember, my post Completion of EMDC" on the IST graduation on the 23rd of May? I spent 75% of my masters in IST. But still it was KTH graduation that really made me feel the completion. Majority of us from EMDC - IST were back in Stockholm after 1.5 years for the graduation. The graduation ceremony took place in Stockholm city hall, where we also had the inauguration ceremony in 2013 August, when we just started our semester in KTH. So much nostalgic.

I ordered the two individual photos taken for 165*2 SEK (35,44 Euro), and transferred the money through my bank from Portugal to Sweden. It cost me 41,60 Euro extra for the foreign transaction fees. It usually charged me this when I transferred money to Sri Lanka. I did not know that they also charge to transfer to Sweden! It is always free to transfer money inside Portugal (also probably inside all the euro-zone countries). I was informed by my bank that it was the charge from the Swedish bank - not my bank. My Portuguese bank does not charge for online money transaction inside Europe. Whatever that is, it seems bank money transfer is not really a good idea.

Now waiting for the official photos to arrive by email. When I came to Portugal in 2012, I just wanted to finish my masters successfully with pretty good results. I did not think that I will stay in Portugal beyond that, though I really had plans to do EMJD-DC, my current PhD program, from the very beginning. But my interest was to do it in KTH. Eventually, my shift changed and I started to like IST and INESC-ID Lisboa. That is how I ended up staying much longer in Lisbon than I initially thought.

In the photo, Prof. Peter Gudmundson, President of KTH, during the welcome reception on the 30th of August, 2013. Same location - same people - nice to be there listening to him again speaking in Swedish and English. Next to Sri Lanka and Portugal, Sweden was my third home for a short period. I often feel guilty realizing that I did not travel much during my 4.5 months stay in Sweden, except Stockholm and Malmo, in 2013. This visit [26th - 29th, May 2015] was a quick recap of my entire stay in a very short time. I will be back for my third visit soon, I believe.

As of now, I don't really have solid plans on my next stop. I will know it when the time comes closer.

Monday, May 25, 2015

Understanding and Protecting Ourselves from Buggy Device Drivers

Given below is the 4th presentation for the NSOM module, which I presented today.