A few things that made my 2018 interesting..

A day in Monaco

2018 is one of my most favorite years. It gave me lots of happy moments. Adhering to the habit I started in 2012, this post summarizes 30 things that made my 2018 interesting.

1. Submitting my Ph.D. thesis

I planned to defend by 2018 October. But it got delayed, and I will defend early next year. However, I have submitted the thesis, and my Ph.D. research is complete. :)

2. Return to Emory University, Atlanta
Emory BMI for a longer stint after my previous short one in 2016. Everything felt the same and different at the same time.

3. A day in Monaco, walking across the entire country
It is trivial to walk across a country if it is as small as Monaco. :)

4. The conclusion of Ph.D. collaborations
It is important to conclude the collaborations on a positive note. As my Ph.D. nears the end, I am happy that my collaborations from URijeka (SDSC) and KAUST (NetUber and Dynam-IX) also came to successful completion.

5. Sleepless nights in Shenzhen, China
When deadlines come, you get to work wherever you are. I ended up working overnights for the successful submission of Óbidos.

6. Shopping in rainy and windy days in Jeju
I was lucky (or was that Jeju, which indeed was fortunate to receive me?). After I left Jeju, in a few months, I learned that Jeju does not offer visa-free entry to Sri Lankans anymore.

7. Best Paper Award at SDS'18
I received the best paper award at SDS'18 for my paper on Software-Defined Data Services (SDDS). It was an exciting moment.

Barcelona, once again!

8. Random Walks in Barcelona, once more!
This is a city I adore. I am getting familiar with this city more and more.

9. Cruise in Zurich Lake
Zurich was still cold in May. Of course not as cold as my last visit for the 2017 New Year's Eve. But still raining.

10. IFIP Networking 2018 with an overdose of caffeine
IFIP is a conference I always wanted to participate. Had the chance to present NetUber there this year, finally.

11. Spending a whole day in Sesimbra Beach
Sesimbra is a beach town close to Lisboa in the Setúbal district. I always wanted to go there. I got the chance finally this year.

12. First journal papers and book chapters of my Ph.D.
I started working on 3 journal papers (Évora, Óbidos, and SD-CPS) and 2 book chapters (SDN for big data and SDN for service compositions) last year. They all got accepted and published this year!

13. The views of French Riviera
Despite the predicted weather that it would be warmer than Barcelona, Nice turned out to be cloudy and colder.

14. Fresh Seafood in Jeju
Watching the show of octopus, squids, and several similar sea creatures being cooked alive for me.

15. Leaving Portugal and EU after 6 years
I don't think I will ever forget those sweet memories of Lisboa. Experiencing 'saudade', as we call it in Portugal.

16. Almada, just across the Tagus River.
Pleasant environment, but busy days.

17. Presenting my CAT at IST
I should have tried to make the IST's CAT and UCL's confirmation as a single presentation. But due to the difference in UCL and IST requirements, I chose to make 2 different presentations. With the CAT and my thesis submission, I have met all the requirements for the thesis (ABD/All But Dissertation).

18. A slow-paced year, quite the opposite of 2017
2017 was way too fast with several migrations and travels. 2018, on the other hand, was more stable and slow-paced. This was the first time during the past 6 years that we stayed home during the new year's eve. Feeling somewhat old.

19. Becoming a morning person in Atlanta
Love those early morning commutes to work while it is still dark and cold outside.

20. Hopping on to random buses in Jeju, hoping to reach the destination eventually.

Rainy days in Jeju

And we did. I even made an algorithm out of this.

21. Bus Commutes from Calçada de Carriche to my lab
My apartment was in Calçada de Carriche, a bit farther from my lab, INESC-ID Lisboa, compared to my previous neighborhoods. Therefore, needed a relatively longer bus commute.

22. Parque das Nações revamped.

Parque das Nações is my favorite spot in Lisboa. It has improved within the past few years with new restaurants, yet maintained its charm. It felt nostalgic.

23. Exploring more of greater Lisboa

Even after 6 years in Lisboa, we realized that there were a few places that we hadn't visited. This year, we visited the Loures region and its big shopping mall for the first time. 

24. Évora after 4 years

Évora has its historical significance. It is also where I was informed that I was selected to the prestigious EMJD-DC program with the grant, 4 years ago.

25. Trying to relive moments in Portugal

As we count the last days in Lisboa, we tried our best to visit our favorite places just one more time, with mixed success.

26. Atlanta Botanical Garden in night lights

The entire botanical garden was shining in colorful lights and music during the December nights.

27. Ponce City Market, Atlanta

I like the interiors of this small-scale shopping mall, especially its food court. It gives strong memories.

28. Assembling Furniture from IKEA

IKEA furniture shopping and assembling them for the first time. A fun experience.

29. Atlanta Christkindl Market

Christmas markets are a significant part of the festival season. This market in Atlanta reminded me of the ones I experienced in EU during the past 6 years, such as, Lisbon, Stockholm, Helsinki, and Louvain-la-Neuve Christmas markets.

30. Poble Espanyol, Barcelona

An open-air museum that summarizes entire Spain. Not too big. But it was a pleasant walk with architectures and some snacks from different Spanish regions.

Every year, I have one new year's resolution - to outperform my previous year. :) I already have high expectations for 2019. I wish you a happy new year everyone. Thanks for reading my list until the end. You may also read the blog posts of all the previous years as well.

Add additional context to SELinux

In this post we quickly look into how we add context for SELinux on our Orthanc binary storage.
$ sudo semanage fcontext -a -t etc_t -s system_u /opt/localdrive/orthancstorage

$ sudo restorecon -R -v /opt/localdrive/orthancstorage

Now we confirm it by the below command:
$ cat /etc/selinux/targeted/contexts/files/file_contexts.local
# This file is auto-generated by libsemanage
# Do not edit directly.

/opt/localdrive/postgres(/.*)?    system_u:object_r:postgresql_db_t:s0
/opt/localdrive/orthancstorage    system_u:object_r:etc_t:s0

Confirm the updates
$ ls -laZ /opt/localdrive/orthancstorage

Fixing SELinux Warnings for Postgres after changing the data directory

We moved the postgres data directory from its default location /var/lib/pgsql/data to /opt/localdrive/postgres.

This started to give lots of warnings in the SELinux audit logs. Postgres service was running fine as SELinux was in permissive mode, albeit giving verbose warnings as below.

$ sudo tail -f /var/log/audit/audit.log
type=AVC msg=audit(1543413248.637:5277): avc:  denied  { getattr } for  pid=5285 comm="postgres" path="/opt/localdrive/postgres/base/16386/PG_VERSION" dev="sdb1" ino=1725 scontext=system_u:system_r:postgresql_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file
type=SYSCALL msg=audit(1543413248.637:5277): arch=c000003e syscall=5 success=yes exit=0 a0=5 a1=7ffef5d9e1f0 a2=7ffef5d9e1f0 a3=1 items=0 ppid=1314 pid=5285 auid=4294967295 uid=26 gid=26 euid=26 suid=26 fsuid=26 egid=26 sgid=26 fsgid=26 tty=(none) ses=4294967295 comm="postgres" exe="/usr/bin/postgres" subj=system_u:system_r:postgresql_t:s0 key=(null)
type=PROCTITLE msg=audit(1543413248.637:5277): proctitle=706F7374677265733A206175746F76616375756D20776F726B65722070726F63657373202020
type=AVC msg=audit(1543413248.638:5278): avc:  denied  { write } for  pid=5285 comm="postgres" name="12730" dev="sdb1" ino=1263 scontext=system_u:system_r:postgresql_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file
type=SYSCALL msg=audit(1543413248.638:5278): arch=c000003e syscall=2 success=yes exit=5 a0=2947210 a1=2 a2=180 a3=50 items=0 ppid=1314 pid=5285 auid=4294967295 uid=26 gid=26 euid=26 suid=26 fsuid=26 egid=26 sgid=26 fsgid=26 tty=(none) ses=4294967295 comm="postgres" exe="/usr/bin/postgres" subj=system_u:system_r:postgresql_t:s0 key=(null)

To fix this, we had to perform a number of steps:

Edit SELinux to point to the current postgres data directory
$ sudo semanage fcontext -a -t postgresql_db_t "/opt/localdrive/postgres(/.*)?"

$ sudo restorecon -R -v /opt/localdrive/postgres

Restart Postgres service
$ sudo service postgresql restart


Now the audit "denied" logs are gone!

Time zone stories of a nomadic student

Back to cooking in Atlanta

I have this love-hate relationship between the daylight savings observed in many countries. It lets me sleep longer and avoid going to work when it is too dark in the autumn and winter. The sunlight starts to shrink with time as we progress into autumn, and the winter time (which is the actual/natural time) kicks in and gives us one more hour to sleep. But as days progress into winter, the sunlight continues to shrink, still making you wake up and go to work in the dark. On the other hand, as spring comes, the days start to lighten up, with more sunlight. Then the summer time starts, to make you wake up earlier, and to give you more light in the nights.

Time zone change is another factor that you often have to deal with when you migrate from east to west or vice versa. It sometimes helps, and sometimes does not. I found that moving towards west usually helped. It helped me when I moved to Portugal from Sri Lanka in 2012, and then again this year when I moved to Atlanta from Portugal. I have been a night person since 2009. This is a habit I initially picked up with my first GSoC, as my mentors and other developers were from EU and US while I was in Sri Lanka. Later in Portugal, my deadlines often kept me up until mornings (conference submission deadlines are often given as 23:59 EDT or 23:59 AoE/anywhere on earth, which is the next day morning until noon in Portugal). This time although I moved to Atlanta in June, I maintained my biological clock synced with Portugal. Therefore, I go to bed at 10.30 pm (3.30 am in Portugal) and wake up at 6 am every day (11 am in Portugal). I leave home around 7.10 am and arrive at the lab at 7.25 am (if the bus is on time) to 7.33 am (if the bus is delayed). I hope to maintain this habit.

Time capsule - Portugal version

Taxi bill found in my pocket

There were many times I left Lisboa to live in other cities. But I knew that we would return, and usually stored our things securely in safe storage before leaving the city. But this was the first time packing everything, throwing away our 6 years of stuff. The city has loads of memories, and it is overwhelming. I have a feeling that the best part of my life so far, was in Lisboa. Not in Colombo, where I grew up.

Migrations consume lots of energy - especially if the migration is permanent. Portugal was my home for the past 6 years. Although I was moving back and forth between countries, I always returned to Portugal within a year. This year leaving Lisboa was first time as in a permanent move. I will of course go back for my Ph.D. defense soon. But not to live there again. At least no plans in the future that I foresee. We were dropping our clothes that are still relatively new into the donation box since we just can take 1 checked-in bag each in the flight. Later I realized, I dropped my metro card inside mistakenly with one of the shirts. Several months later in Atlanta, I found the taxi receipt from Lisboa in the pocket of my jacket, which I luckily still did not wash. On the taxi to the airport, I had mistakenly dropped a new umbrella and a water bottle from the side pockets of my bag. Leaving a country, throwing away almost everything we accumulated over several years is in fact somewhat a painful experience that we got used to. In fact, that explains our minimalist lifestyle during the past few years in EU.

Memories of Porto, still strong after 6 years

Sometimes we want to go back to visit a few of the places that left strong and pleasant memories in the past. However, it is not always easy to recreate and relive those memories. I wanted to spend some time in Porto before leaving Portugal this year. It was always in my list since we last visited in 2013. However, due to time limitations, we could not make it this year. It will remain forever as a memory of 2013 summer, until I manage to visit it in the future. This takes me back to the memories of Kista and Farsta in Stockholm. I stayed in Kista for my 2013 falls semester in KTH for my EMDC masters program. I used to visit Farsta frequently those days for a walk and for the shopping mall. Farsta also had a Hindu temple. After I left Stockholm and returned to Portugal, I visited Stockholm twice. Once for my masters graduation at KTH and once when I returned from ACRO summer school from Karlstad. I managed to visit Farsta during my first visit. But I missed Kista during both of my visits. Kista is not much of a fun except the shopping mall which also hosted our student accommodation as well as our Kista campus (far from the main KTH campus).

Memories are the best part of a travel. I sometimes like to travel to new places. Often I just want to go back to a place that I previously enjoyed and loved. Even if I go back, the same places do not always give the same memories. Things change. Places also change. Thus our experiences.

Installing Standalone Viewer of OHIF

You can install an OHIF Simple Viewer by following the below steps:

1) Install Meteor
$ curl https://install.meteor.com/ | sh


2) Clone the Viewers repository
$ git clone https://github.com/OHIF/Viewers.git

 
3) Go to the SimpleViewer directory
$ cd Viewers/StandaloneViewer/StandaloneViewer/


4) Run the Standalone viewer
$ METEOR_PACKAGE_DIRS="../../Packages" meteor --settings ../../config/dcm4cheeDIMSE.json

Make sure that your JSON file consisting of the URLs of DICOM images (i.e., series instances) is placed inside the directory Viewers/StandaloneViewer/StandaloneViewer/public
as in
Viewers/StandaloneViewer/StandaloneViewer/public/big.json
 
5) Access the viewer from the browser.
You may access it from localhost:3000 followed by the url of a JSON file consisting of Dicom images.
http://localhost:3000/?url=https://raw.githubusercontent.com/OHIF/Viewers/master/StandaloneViewer/StandaloneViewer/private/testData/PTCTStudy.json
or access your local json files consisting of the DICOM instances, as in,
http://localhost:3000/?url=big.json

Restaurant Scams in Atlanta

This weekend we had dinner at an Indian restaurant. My bill came as 37$ for the food we consumed for 29$ (11+ 15 + 3). The additional 8$ includes the sales tax and 20% service charge. (We were a group - but asked for individual bills). Since there was already a 20% service charge, I put 0$ on the tip. I clearly also indicated 37$ as the total in the bill (37 + 0.00 = 37).

Initially, my bank showed 37$ as pending. Now it shows 43$ as the completed amount. Mistakenly, or intentionally, the restaurant charged 6$ as the tip. I have a hard time to believe this is an honest mistake, since I also wrote down the total amount as 37$, after putting 0$ under the tip.


Also, a 20% service charge for a lousy service was not something I was planning to do. The food was below average too - and overpriced. Of course, from the menu, it looked 29$ for two, and with sales tax and 20% mandatory service charge, it came to 37$. Add the 20% service charge, why would I pay another 6$ to make a 40% tip in total for this below-average experience? Seriously?!

The line left in the bill in addition to the auto-charged 20% service fees is a classic case of a double-tipping scam, a variant of gratuity scams. That is, tricking you to tip more, after they have already made a mandatory tip of 20% from you!

I have disputed this additional 6$ charge with my bank. The bank said, generally, they charge for the disputes, though they can make it free for me this time since this is my first dispute. However, they recommended to contact the restaurant and resolve this dispute with them.


So I called them. Judging from the voice, the first guy to answer the call was the waiter who served us (the second one). I explained to him my situation. He insisted I visit the restaurant with the bill to get a refund. He pretended that he could not find their copy of the receipt, while I was waiting for him searching through the records. He also verified what I had, where we sat down, and how much each cost, etc. Like security questions. :P I told him "I am busy to come to the restaurant to get the refund." when he repeated as if that is the only way. He said, "come on, you have time to call for 6$, and you tell me you are busy?" sarcastically. His intention was to shame me and make me give up. Now I realized he was the one who intentionally made the 0 to 6$ to get the 6$ for himself. I told him that I have already disputed this with the bank. Then he said, "Ok, give me your name and phone number. We will call you back if we find out what you say is correct". Then he also asked me to spell my name! Come on. :D He was also very unapologetic.


Within a few minutes, they called back. This time it must be a senior or a manager. He was polite and apologetic, and he accepted that it was their mistake. He said, "Your zero gave the one who entered to put a 6" (not sure what he meant. Did he mean the employee was a fraud or was it an honest mistake? It was unclear). He said they would send me a cheque for the 6$.

Update: I received a cheque from the restaurent on the 28th/Friday, 3 days from the 25th/Tuesday since I complained. This issue is resolved!

Anyway, lesson learned. Next time, I would not put 0.00 $ and write the same amount down, if I choose not to tip. I always tip. But not when a 20% service charge is already added to my bill. I would make sure just to cut the section standing for "tip" if I were to give no tip!

Luckily I paid attention. I am not sure how many customers lost their money to this scam.


Summary

1. Keep your receipts safe, especially when the waiter takes your card away from your view, and especially when there is a potential for them to seek a tip from your card.

2. Check your bank accounts online every day, and keep track of the balance, to make sure there are no weird transactions.

3. Report to the bank instantly if you find something weird.

4. Be extra vigilant if you are a tourist. You cannot spend time and money making international calls, and your bank may have limited control over a foreign transaction. Also, if your transaction is in a different currency, it will make the things even more complicated. If I were not living in Atlanta, even if they send me a cheque, I won't be able to receive it anyway - giving me no choice other than to give up.

Configuring Orthanc with Postgres backend with a network data directory

So we hve configured Orthanc with a Postgres backend. To support a large-scale data store, we mapped a network directory as the data directory of Postgres. Then we configured Orthanc to have Postgres as its backend data store, instead of its default SQLite backend, using the Postgres plugin. There is also an option for a MySQL/MariaDB backend, which we found not stable with MySQL in a network directory.

However, since we have the configured Postgres in a network directory, we have to make sure everything is running fine. Unfortunately, when we reboot, often the network directory does not mount on its own. Therefore, despite our configuration to start Postgres and Orthanc at the boot time, they both fail.

Data directory unaccessible → Postgres fails to start. Postgres failed to start → Orthanc fails to start.

We have to configure the below services in Centos, following the same order.

1) Postgresql

$ sudo systemctl start postgresql

$ sudo systemctl enable postgresql

$ sudo systemctl status postgresql

● postgresql.service - PostgreSQL database server

   Loaded: loaded (/usr/lib/systemd/system/postgresql.service; enabled; vendor preset: disabled)

   Active: active (running) since Mon 2018-09-17 14:46:41 EDT; 11min ago

 Main PID: 2655 (postgres)

   CGroup: /system.slice/postgresql.service

           ├─2655 /usr/bin/postgres -D /opt/pacs/postgres -p 5432

           ├─2657 postgres: logger process

           ├─2701 postgres: checkpointer process

           ├─2702 postgres: writer process

           ├─2703 postgres: wal writer process

           ├─2704 postgres: autovacuum launcher process

           ├─2705 postgres: stats collector process

           ├─2754 postgres: postgres orthanc ::1(48534) idle

           └─2755 postgres: postgres orthanc ::1(48536) idle

Sep 17 14:45:59 HOST.NAME systemd[1]: Starting PostgreSQL database server...

Sep 17 14:45:59 HOST.NAME pg_ctl[2652]: pg_ctl: another server might be running; trying to start server anyway

Sep 17 14:46:41 HOST.NAME systemd[1]: Started PostgreSQL database server.

2) Orthanc

$ sudo systemctl start orthanc

$ sudo systemctl enable orthanc

$ sudo systemctl status orthanc

● orthanc.service - Orthanc DICOM server

   Loaded: loaded (/usr/lib/systemd/system/orthanc.service; enabled; vendor preset: disabled)

   Active: active (running) since Mon 2018-09-17 14:47:14 EDT; 8min ago

     Docs: man:Orthanc(1)

           http://www.orthanc-server.com/

 Main PID: 2753 (Orthanc)

   CGroup: /system.slice/orthanc.service

           └─2753 /usr/sbin/Orthanc /etc/orthanc/orthanc.json

Sep 17 14:47:15 HOST.NAME Orthanc[2753]: W0917 14:47:15.399045 ServerContext.cpp:167] Reloading the jobs from the last execution of Orthanc

Sep 17 14:47:15 HOST.NAME Orthanc[2753]: W0917 14:47:15.399776 JobsEngine.cpp:281] The jobs engine has started with 2 threads

Sep 17 14:47:15 HOST.NAME Orthanc[2753]: W0917 14:47:15.400023 ServerContext.cpp:293] Disk compression is disabled

Sep 17 14:47:15 HOST.NAME Orthanc[2753]: W0917 14:47:15.400050 ServerIndex.cpp:1437] No limit on the number of stored patients

Sep 17 14:47:15 HOST.NAME Orthanc[2753]: W0917 14:47:15.400490 ServerIndex.cpp:1454] No limit on the size of the storage area

Sep 17 14:47:15 HOST.NAME Orthanc[2753]: W0917 14:47:15.400995 LuaContext.cpp:103] Lua says: Lua toolbox installed

Sep 17 14:47:15 HOST.NAME Orthanc[2753]: W0917 14:47:15.403966 main.cpp:848] DICOM server listening with AET BMIPACS on port: 4242

Sep 17 14:47:15 HOST.NAME Orthanc[2753]: W0917 14:47:15.404382 MongooseServer.cpp:1087] HTTP compression is enabled

Sep 17 14:47:15 HOST.NAME Orthanc[2753]: W0917 14:47:15.405872 MongooseServer.cpp:1001] HTTP server listening on port: 8042 (HTTPS encryption is disabled, remote access is allowed)

Sep 17 14:47:15 HOST.NAME Orthanc[2753]: W0917 14:47:15.405915 main.cpp:667] Orthanc has started

To clean the data from Orthanc entirely

 

The easy way is to drop the Orthanc database.

First connect to the postgres client:
$ psql -U postgres
Password for user postgres:
psql (9.2.24)
Type "help" for help.


postgres-# \l
                                 List of databases
   Name    |  Owner   | Encoding |  Collate   |   Ctype    |   Access privileges
-----------+----------+----------+------------+------------+-----------------------
 dcm4chee  | dcm4chee | UTF8     | en_US.utf8 | en_US.utf8 |
 mytest    | postgres | UTF8     | en_US.utf8 | en_US.utf8 |
 orthanc   | postgres | UTF8     | en_US.utf8 | en_US.utf8 |
 postgres  | postgres | UTF8     | en_US.utf8 | en_US.utf8 |
 template0 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
 testtest  | postgres | UTF8     | en_US.utf8 | en_US.utf8 |
(7 rows)

Then go to another database to be able to drop the Orthanc database
orthanc=# \c mytest
You are now connected to database "mytest" as user "postgres".

mytest=# drop database orthanc;
ERROR:  database "orthanc" is being accessed by other users
DETAIL:  There are 2 other sessions using the database.

Yes, first we need to stop Orthanc!

[root@researchpacs postgres]# systemctl stop orthanc

Now drop the database. 

mytest=# drop database orthanc;
DROP DATABASE
mytest=#

Create the database again.

mytest=# create database orthanc;
CREATE DATABASE

Now you may start Orthanc again!

[root@researchpacs postgres]# systemctl start orthanc



You can also access your data via a browser, using the HTTP Port:

http://HOST.NAME:8042/app/explorer.html

Configuring Kong API Gateway for your Service Endpoints

Kong provides a complete documentation on its installation and a quick-start guide to start using it, such as configuring a service in Kong. 

In this post we will briefly look into configuring Kong using its Docker container as an API gateway for your backend services. This document uses Kong version 0.14.0-alpine, which is the current latest version of Kong. Kong can be configured with Postgres or Cassandra backend for its persistent storage. Here we will configure Kong with Postgres.

Install and Start Kong with Postgres

You have 2 options. If you want to get it done quick, I recommend choosing the option #2 - installing via Docker containers.

1. Download Kong's native installation for your operating system

Download and install Kong for your respective operating system 

Configure with Postgres:

$ psql -U postgres

postgres=# CREATE USER kong; CREATE DATABASE kong OWNER kong;

Run the Kong migrations:

$ kong migrations up

Start Kong

$ kong start

You may choose to start with verbose logs:

$ kong start -vv

You may need to create a kong configuration file to load Kong with custom configurations:

$ sudo mkdir /etc/kong

$ sudo touch /etc/kong/kong.conf

Now your Kong is running. Confirm that by,

$ curl -i http://localhost:8001/

 

2.  Install Kong via Docker containers.

We have our own "kong-ldap" repository with scripts that will install and configure Kong with Postgres in a container. 

$ git clone https://github.com/pradeeban/kong-ldap

$ cd kong-ldap

$ sh buildRun.sh

Optionally, you may also use these scripts to configure OpenDJ LDAP directory as a Docker container a well, connected to Docker. However, the commands are commented out - therefore you will need to uncomment the relevant commands in the script buildRun script to get it working.

Kong has two interfaces. One is the user-facing interface and the other one is the admin interface. The admin interface by default listens to the port 8001 and should not be exposed to the public. The admin interface is used by the administrators to create and configure the routes to the services in the Kong API gateway. The user-facing interface by default listens to the port 8000. It is exposed to the public, and the users can consume the services defined by the admin previously using the user-facing interface. The user-facing interface is completely separated by Kong from the admin interface.

Configure Kong for your Services

Configuring Kong as an API gateway for your services is a 2-step procedure, starting from Kong 0.14. Previous versions of Kong provided a unified approach through its "api" objects, which is current depreciated and replaced by two entities known as "services" and "routes".

Make sure that you have Kong up and running. Then,  execute the below two commands from the server that hosts Kong.

First, you need to define a "service" in Kong to each of your service/API groups (i.e., your backend applications or web services).

1) Create a Kong Service using the Kong Admin API

$ curl -i -X POST --url http://localhost:8001/services/ --data 'name=radiology' --data 'url=http://172.20.11.223:9099/services/v4/TCIA/query/'

Above, we assume your web application is hosted in http://172.20.11.223:9099/  and it has a set of services under the base path /services/v4/TCIA/query/. We pass the complete url of the backend of the original service deployments through the "url" flag, as shown above. We create a service named "radiology" in Kong for these services, using the Kong admin interface for the creation of services. Here, we assume Kong is hosted in localhost, with its admin interface using the default port option of 8001. /services/ let you create/modify the service definitions in Kong.

Second, you should add a "route" to the "radiology" service that you created.

2) Add a Route to the Kong Service

$ curl -i -X POST --url http://localhost:8001/services/radiology/routes --data 'paths=/radiology'

 As shown above, we use the KONG-ADMIN-INTERFACE/services/SERVICE-NAME/routes to configure the routes to the service that we defined above. Here the SERVICE-NAME is "radiology" as we defined above in the previous step. The "paths" flag indicate the paths that Kong should match to the service that we defined in the previous step.

Accessing your services via Kong

Now, you may access your services via Kong

http://172.20.11.222:8000/radiology/getImage
the same way you access it directly
http://172.20.11.223:9099/services/v4/TCIA/query/getImage

Here, we assume that you have deployed Kong in a server with a public address http://172.20.11.222 and your services are in http://172.20.11.223.

You may of course do additional tasks such as rate limiting, request/response transformations, logging, and analytics with the Kong API Gateway. You may refer to the Kong documentation on pointers to achieving this.

Export Confluence HTML to Github Wiki

Confluence allows its HTML pages to be exported in a single zip. There is a Confluence-to-Github-Markdown tool that allows automatic export of these HTML files to markdown .md files with a single command from the core directory:

$ confluence-to-github-markdown

Now you can create separate pages in the GitHub wiki with these md files. However, there are a certain limitations that need to be manually handled to complete the process.

1. Image. GitHub wiki does not support uploading images. Images are posted via URLs in the wiki. Therefore, currently we need to upload the images to a public location and access the URL from there.

2. Metadata. The tool also converts metadata (which is actually good I think) to md. We need to manually remove them wherever unnecessary instead of showing them as plain text in the GitHub wiki.

3. The converter tool creates the name of the md file from the html title. If you have ' or any special characters in the title of your html, it will fail to convert and throw js errors. To fix, change the title header from something such as "Rakshak’s RESTful API" to "Rakshak RESTful API" before executing the converter tool.

4. Internal links (links to named anchors in the markdown) are not handled by the converter.  The anchors must be manually added.

5. Links to other pages in the confluence wiki are preserved as it is, rather than having them as a relative link. Therefore they need to be replaced by the respective new link in the markdown wiki.

6. Video embedding is not supported in Markdown. Therefore, the converter tool simply ignores any video files embedded in the HTML. To fix this, in the final wiki pages, manually add the videos as a simple HTML links as in, [screencast](https://www.youtube.com/embed/S8juo0Dx68I).

Installing Gravitee API Gateway

Gravitee running

Configuring and Starting Gravitee API Managemant Gateway

First install and start the dependencies:

1) Mongo
$ mongod

2) Elastic search
$ cd elasticsearch-6.3.2/bin/
$ ./elasticsearch




Starting Gravitee API Managemant Gateway
Download and extract the project zips of graviteeio.

First, Start the API Gateway
$ cd graviteeio-full-1.18.1/graviteeio-gateway-1.18.1/bin
$ ./graviteeio

Now accessing http://localhost:8082/ should give you the message:
No context-path matches the request URI.


Configure graviteeio-gateway-1.18.1/config/gravitee.yml
By default, the gravitee data is stored in the mongo database gravitee.
You can change these default configurations.


Then, start the Managemant API.
$ cd graviteeio-full-1.18.1/graviteeio-management-api-1.18.1/bin
$ ./gravitee

Now you can access http://localhost:8083/management/apis/ which should give an empty response [ ].

By now, the gravitee database in MongoDB should have the below collections:
events, ratelimit - for the API gateway
audits, metadata, roles, views - for the Management API

More configuration details such as using an LDAP provider and configuring OAuth2 Authentication can be found from the official documentation.


Next, start the Portal.
$ cd graviteeio-full-1.18.1/graviteeio-management-ui-1.18.1

You have the choice of starting the Portal with Python3 or Node.

Let's go with Python3.
$ python3 -m http.server
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...

Now you can access the portal from http://localhost:8000/#!/


You may further configure the portal to fit your requirements.

Configuring and Starting Gravitee Access Managemant

First, start the gateway
$ cd gravitee-am-gateway-standalone-2.0.4/bin/
$ ./gravitee

Now you should get an empty response to http://localhost:8092/
Now, run the AM Managemant API.

$ cd gravitee-am-management-api-standalone-2.0.4/bin/
$ ./gravitee

Now, accessing http://localhost:8093/management/domains/ should give you an empty response.


Finally, let's start the last component of our ecosystem.
The AM Managemant UI, which is a client-side Angular application.

The directory of the Managemant UI is at,
$ cd gravitee-am-webui-2.0.4/

You will need an HTTP server such as Apache or Nginx. Then you may configure the Management UI following the documentation.

The Atlanta Algorithm

As with many other cities in the US, Atlanta has a public transportation network (known as MARTA), which is a joke. In fact, the metro is ok. But my issue is with the buses. Unfortunately, most of Atlanta is not also walkable. Lack of proper sidewalks in the residential neighborhoods. Life in Europe has taught us that it is not necessary to drive, and it is ok to use the public transport. :D Given this state, we choose to live close to my university. 20 minutes by walk.

There is also a bus stop close to both my apartment and my lab. A bus (#6) passes by every 30 minutes, connecting my home to my lab in the university. I usually walk to my lab, and return, by foot. But since I have a free monthly public transportation pass from the university, I decided to use it.

"MARTA On the Go" a mobile app offers a real-time schedule of the bus, unlike Google maps which shows a static schedule. Today I checked the bus time just before leaving home. The bus was on its way. It said "6 minutes late" with the current location. MARTA On the Go does not show the estimated arrival time at each stop with this dynamic information. Rather, it shows the current location of the bus (that means, you should have an idea of the road map, or you need to use it in conjunction with another application such as Google maps) and the current delay. While this app does not solve the inefficiency of the public transport, it certainly mitigates the uncertainty and help plan the timing better.

Before leaving home, I check the app to confirm the arrival time of the bus. However, the timing is not accurate. The bus usually comes with a time interval [-3:5], that means 3 minutes earlier to 5 minutes later. But now for the third time in a row, the bus came 10 minutes late. To ensure I do not miss the bus, I always arrive 3 - 4 minutes earlier. Therefore, I leave home 7 minutes earlier than the bus' arrival time at the bus stop (since I need to walk a bit to reach the bus stop).

Sometimes a bus appears without showing in the app. Sometimes the bus shows in the app in the opposite direction. Sometimes a bus shows up in the app, but it is in fact out of order. So the app is useful. But not complete.

Every city's public transportation system teaches me some new life lesson. I learned something in Jeju. Now in Atlanta. I like these life lessons - I do not underestimate them. Trust yourself, instead of trusting an unreliable external entity!

Me and Long Flights

I enjoy watching movies on the long flights because they have English subtitles for foreign language movies. Sometimes I just fall asleep in the flight while attempting to watch a movie. :D

In 2015 I got my luggage delayed for the first time. The delay was by a day, and it was in PHX when I attended my first SDS conference. During the return trip, when I reached SJC after completing the trip, I was informed that there is a flight delay that might lead me missing my connecting flight in LAX. Hence, I was given a free shuttle ride to SFO and sent to LIS in an alternative route.

I love travels. I am not entirely sure whether I love or hate airports. When I look back, airports always left me with positive memories. But while waiting for a delayed flight, or when I had to rush for a connecting flight, it was rarely a pleasant feeling.

My "CAT"/ Ph.D. Proposal

Today I presented my CAT, " Software-Defined Systems for Network-Aware Service Composition and Workflow Placement". I should have presented the CAT much earlier. However, it was delayed, and now I have presented it just a few months before my final defense. I received a grade of 18/20 for the CAT. The presentation slides are attached below.

Software-Defined Systems for Network-Aware Service Composition and Workflow Placement from Pradeeban Kathiravelu

The Conference Survival Guide

SoCPar 2010, Paris

I enjoy attending conferences. I never attended a conference when I did not have something to present. This is usually because of the funding, as it is unlikely for me to get funded to a conference when I am not presenting my work. So far, WWW2011 is the only conference that I attended without having a paper to present. Conferences help me fine tune my research, make new friends and build a network. I still vividly recall my first conference, SoCPaR 2010. I was presenting our paper, and luckily for me, I had 3 of my best friends with me. I was not alone. We all were co-authors of the paper, a team effort during our BSc times. I recall presenting the paper confidently and discussing with other researchers. Prof. Ajith Abraham of MIRLABS, who was organizing the conference remarked that he remembered us, a 4-member team arriving in Paris all the way from Sri Lanka to present our paper. It was a nice experience to travel to Paris, my first overseas trip. It was a cold winter, full of snow. The conference was held in Université de Cergy-Pontoise. I met two researchers from the same university in latter conferences, bringing back memories from 2010. 

Be prepared for the delayed luggage

Interestingly, the second conference I presented my paper was again in Paris in 2014. This time, it was MASCOTS, and this was the first paper of my MSc, and also the first paper as the first author. The other conference as part of my MSc research was UCC'14 in London. I met my friend there after a long time, and we had some good chat!

I love visiting different countries and various cities. Attending conferences also offers this opportunity. Moreover, the conferences immediately give you some partners to travel with. You meet some random researchers and make friends with them - because of the shared research interest (or purely because you are from the same university, country, or have something in common other than the research interest). For the rest of the conference, you can explore the city with them.

I try my best to attend all the sessions at the conferences. However, sometimes it is inevitable that I miss one session. Especially the earliest one on the day following my presentation. This is because sometimes I feel tired after the long presentation and its preparations and oversleep the following day. I also enjoy the coffee breaks, lunch times, and the gala dinner. The ideal time to make friends. You are lucky if your presentation comes on the first day. You can relax and enjoy the remaining talks/sessions without having to check your presentation once in a while.

I have attended several conferences during my Ph.D. As a result, I have traveled to several cities: Tempe, AZ, USA (IC2E'15); Berlin, Germany (IC2E'16); San Francisco, CA, USA (AMIA'16 and ICWS'16); Rhodes, Greece (CoopIS'16); Valencia, Spain (SDS'17); Munich, Germany (VLDB'17); Barcelona, Spain (SDS'18); and Zurich, Switzerland (Networking'18). Sometimes, I have one or two additional days following a conference. I use these days to travel to a near-by country or a city.

IFIP Networking'18 is my last conference as a Ph.D. student. It was also my best conference experience so far. I thoroughly enjoyed and actively participated in IFIP where I presented my paper, NetUber. I attended all the sessions. Conferences make me happy. And caffeine helps me remain super-active throughout the whole day of sessions, followed by evening and nightly walks. An overdose of caffeine made me super-active throughout the IFIP sessions. However, after I returned from the conference I became a zombie, with the withdrawal symptoms. :D

Conferences always leave me with good memories - no exceptions so far. They give me more knowledge, and also other experiences with travels. I thank all the conference organizers and volunteers who make sure that we all have good experience attending the conference.

Moving Bits with a Fleet of Shared Virtual Routers

Today I presented my paper titled "Moving Bits with a Fleet of Shared Virtual Routers" at IFIP Networking 2018 in Zurich. The presentation was very interactive. This was my work from KAUST. It is nice to be back in Zurich.

Moving bits with a fleet of shared virtual routers from Pradeeban Kathiravelu

Abstract: The steady decline of IP transit prices in the past two decades has helped fuel the growth of traffic demands in the Internet ecosystem. Despite the declining unit pricing, bandwidth costs remain significant due to ever-increasing scale and reach of the Internet, combined with the price disparity between the Internet's core hubs versus remote regions. In the meantime, cloud providers have been auctioning underutilized computing resources in their marketplace as spot instances for a much lower price, compared to their on-demand instances. This state of affairs has led the networking community to devote extensive efforts to cloud-assisted networks --- the idea of offloading network functionality to cloud platforms, ultimately leading to more flexible and highly composable network service chains.

We initiate a critical discussion on the economic and technological aspects of leveraging cloud-assisted networks for Internet-scale interconnections and data transfers. Namely, we investigate the prospect of constructing a large-scale virtualized network provider that does not own any fixed or dedicated resources and runs atop several spot instances. We construct a cloud-assisted overlay as a virtual network provider, by leveraging third-party cloud spot instances. We identify three use case scenarios where such approach will not only be economically and technologically viable but also provide performance benefits compared to current commercial offerings of connectivity and transit providers.

The Road to Monaco

The Monte Carlo Casino

Finally, I visited Monaco! It was very similar to my trip to Andorra in 2015. When I visited Barcelona for an EMJD-DC Spring Event, I had an extra day. So I took a bus to Andorra and spent a whole day there before returning to Andorra. This time I had a free day after the conference. I also found a cheap return flight ticket (50 Euro Vueling) to Nice from Barcelona. From the Nice airport, you can get a bus (no 110) to Monaco. Purchase the tickets at the ticket office as the bus driver does not sell the tickets.  A return trip to Monaco costs 33 Euro. One way costs 22 Euro. So I purchased a return ticket.


Once you have your bus ticket, proceed to catch the bus at the bus terminal #3 right outside the airport. The bus is hourly. There is also a bus that goes to Menton with the same number (no 110). At least during our trip, two buses left at the same time, ours with the destination to Monaco, and the other to Menton (a city in French Riviera that comes right after Monaco). Make sure to get on the right bus. The bus drivers speak good English, and you will be fine.

Monaco is the smallest UN member nation since Vatican is not a UN member state. Before the journey started, the bus driver proceeded to ask from each of us where are we going. By doing so, he made sure that no one missed their bus stop. That is a pretty smart thing to do. I got down at the Monte Carlo casino with many others. The bus goes until the Monte Carlo beach. There was a tourist information desk in a short walking distance from the bus stop. There was a hop-on-hop-off bus. However, I did not take that and proceeded to walk across the entire country. Thanks to its small size, I managed to create a record of my own to have walked across a country for the first time! :D I discourage anyone from taking the hop-on-hop-off bus. It goes in parallel with the coastal line and does not even go to the Jardim Exotica (7 Euro entry fee). I easily walked to more places than what the bus covers. Also, why waste money on something useless? Monaco (combined with its towns such as Monte Carlo and Monaco Ville is entirely walkable). It also has a complete transport network of buses consisting of public buses.

Monaco is a rich country. A small 19 sq meter studio apartment costs 2,500 Euro per month as rent, excluding the utilities, as I found from an apartment agent advertisement in Monte Carlo. I paid 500 Euro for a studio of the same size inclusive of all the costs in Lisboa. I know people earn in Monaco much more than in Portugal, while not having to pay taxes. I had a good meal (bread, starters, entry, main dish, and dish), a menu option for 29 Euro (34 Euro including the glass of wine). It was a very crowded simple restaurant. Must be one of those on the cheaper side of the price spectrum as far as Monaco is concerned, I guess. I also found other restaurants that are more expensive.

There were several casinos including the Monte Carlo Casino and Cafe Paris Casino near the bus stop. Several shops selling luxury brands, and there was also a very nicely illuminated shopping mall with many such stores, giving it a golden feeling. As far as I noticed they did not have our cheap shops such as Zara and H&M. There was also a nice park with walking trails and a Japanese garden. I paid a visit to both. I avoided the Jardim Exotica as its entrance is not free. I visited the cathedral. Ok, if you are still reading the blog post, by now you would have identified that Monaco is not just casinos and shopping malls.

I walked across the beach until the end. I was thinking - where to get the bus now?! Luckily I found the bus stop of Monte Carlo Beach easily as I step out of the beach, completing my long walk. So did not have to spend time trying to locate it, or returning to the stop I arrived at. While waiting for the bus, a very old lady (in her 80s) came to me and asked me something in French. I told her that I did not understand. She immediately switched to English and asked me to tell her when the bus number 6 arrives. We immediately proceeded into a long conversation. When I asked her how many languages she can speak, she proudly answered: "oh, many." She happily recalled her life with three passports (born in France, 1 km from the bus stop, and having a husband with a Switzerland mother and a Switzerland passport - and thus having passports from Monaco, France, and Switzerland). Since she lives in Monaco, she has a Monaco passport and does not need to pay taxes. "That's the good thing about this," she claimed.

She also asked about what I am doing in Portugal. When I said I am studying, she proceeded to ask my age, and gave a look with a face, "Dude, that is too old of age to study. Did you fail your school or something?" lol. I tried to explain to her what is a Ph.D. I am pretty confident she knows what that is, given her exposure to the world (She has traveled as far as Hong Kong and her husband worked as a hotelier). But probably due to her difficulties in hearing, she did not quite catch what I said. Before I tried to explain her, the bus number 6 arrived. I stopped the bus for her and told her her bus is here. She thanked and greeted me good luck and proceeded to board the bus. I like nice people. They give you good memories that are hard to forget.

I returned to Nice to catch the last flight to return to Barcelona. I also saw the same two couple (looked like a young couple with the lady's parents) that came with me on the bus from Nice to Monaco. Previously on the bus, we shared a brief smile. The young lady came to me and asked something in Spanish, "blablabla, Moh-nuh-koh?" I understood she was asking something about Monaco. But not sure what is that she is asking about. I smiled and replied, "Sorry, I do not understand." I hate to say that. But unfortunately, I am not great at picking up with natural languages. :( She replied, "Monaco - good?" while giving a thumbs-up gesture. I said, "ah, yes, yes, good." giving back a thumbs-up, and asked her back, "And you? How was it?" She replied, "Yes, good! good!!" with a big smile. We all boarded the flight. I am happy to know that it is not just me doing this crazy travel of a same day return flight to Nice from Barcelona just to go to Monaco and return on the same day. There were 5 of us, including the young and old couple. I am not alone in this world. We are all similar in our uniqueness.

Airport Stories

The beautiful French Riviera (Côte d'Azur)

I am waiting for the flight to return to Lisboa in the Barcelona airport. Yesterday was a crazy day. First time in my life I had a return flight on the same day, from/to Barcelona and Nice. And also a non-stop walk across Monaco, stopping only for lunch. The French Rivera is a beautiful region. I enjoyed the view from the bus as it passes by with the view of mountains covered by clouds and glimpses of the Mediterranean sea with yachts. Barcelona airport does not seem to have free wifi in the airside. At least where I am sitting, has not even a slight trace of public wifi. I write this blog post sitting on the seats on gEdit, saving it to post when I return to the civilization where I have the Internet access.

The Jeju Algorithm

I like to travel with minimal or random plans. Traveling to Jeju was remarkable. But it did give some challenges in unplanned random moves, due to the language barrier. Google maps were showing in Korean. It is easy to read sentences in foreign languages as long as the language is written in a script that you can read (such as a Latin script). But that is not the case with Korean.

The locals in Jeju were amicable and came towards to help us with directions even without we asking. But unfortunately, most of them were able to speak only Korean. Then, there are information screens in every other bus stop. Though they are supposed to work in many languages including Korean, English, and Chinese, the touch screen was not working correctly, and it was nearly impossible to change the language in many of those machines. Buses had the information in English, such as the next stop, and the stop after that.

We had to go to a place in the main street. We know that it is a straight line. But several buses were going on the road along the direction. We decided to get on a random bus. We planned to stay on the bus as long as it goes straight. But unfortunately, at the next stop, it turned to a side road in the right. We got down and returned to the main street, and boarded another bus in the next stop. This bus also turned left after two stops. Finally, we asked a foreigner. He did not know the location that we are talking about. But when I told him that we need to go in a straight line, it is probably number 105 (not the actual number he mentioned. I just forgot what he said at this time. So let's assume that is 105 for the sake of this blog post). He warned us that he is not so sure. Then we were waiting. Another bus came. But not number 105. We decided to continue our Jeju algorithm. Our Jeju algorithm works with limited knowledge and certain assumptions. The path is definitely in a straight line, and there is at least one bus that reaches the destination. There is, of course, no direct bus between our origin and destination. We boarded this bus though it is not number 105. Luckily, the bus indeed took us to the destination. Not bad. We reached the destination by using three buses. Ideally, we should have used only two buses. But as our algorithm is not optimized, we ended up with three buses. :D Our return trip was tuned with the knowledge gained, and we returned merely with just one transit (using two buses).

Though this was not, of course, an optimal experience (we wasted some time and money for the bus fare for each leg of our trip). But it did leave us with some exciting memory to talk about for the years to come. :D

How to time-travel?

Time traveling is an exciting concept that has been portrayed in several fiction, literature, and movies. While I am not sure about its feasibility, at least we can simulate the feeling. :D

Let's try, step by step, to travel back in time.

The first step, pick a time and location that you like to time travel. It should be a location that you have been in the past time. It should have given you strong memories. It does not have to be happy memories. Not all the sad memories are necessarily terrible.

Have you decided a place and a time? I have for myself, chosen a few pairs of locations and years:  2017 in Louvain-la-Neuve, 2016 in Atlanta, 2015 in Rijeka, 2013 in Kista, 1987 until 2012 several parts of Colombo, and 2012 in New Delhi.

Now for this to work without causing confusion to you, you should not have visited the same places in between. Otherwise, the memories collide. :D For example, I haven't visited Rijeka after 2015 and Kista after 2013.

Also not all the places will take you back in time giving a time-traveling atmosphere. I was in Stockholm for the 2nd part of my EMDC MSc program in 2013. But I returned to Portugal for my thesis in 2014. However, it did not feel the same anymore. My friends did not return to Portugal with me. So make sure to bring the same people with you when you plan your time traveling. Might be a good idea to wear the same clothes too, to bring back the exact moment. :P

Next, are you back in your destination [location, time], such as [Rijeka, 2015]? Great. Make sure to go to the same restaurants, beaches, and universities that you frequented. Use the same mode of transport. If possible, eliminate anything new. Like, if you never owned a laptop in 2015, don't bring a laptop with you. :D

Great! Music plays a huge role in our memory. Listen to some songs or watch some music videos of the time. It works best if you did not listen to these songs in between. Otherwise, the original memories you allocated to the song might have faded away.

I have picked the below song, which was always playing in Rijeka wherever I went in 2015. I also did not listen to this song afterward. So it is my song of 2015 summer in Rijeka. :P

It is also important to create memories in the first place, to be able to recall them. Making memories is not easy. We don't remember every minor detail of our daily life. But we do remember when we do something remarkable or different. As a thought experiment, during my recent visit to Barcelona, I had Udon at the nearest restaurent for all 3 dinners. I am sure I will remember this forever. :D

Software-Defined Data Services: Interoperable and Network-Aware Big Data Executions

Today I presented my work titled "Software-Defined Data Services: Interoperable and Network-Aware Big Data Executions" at the Fifth International Conference on Software Defined Systems (SDS), in UPC Barcelona. This is my third time in Barcelona. However, this is the first time I am on my own here without my friends, as my previous visits were for the EMDC and EMJD-DC events in 2013 and 2015. This is my 4th time attending an SDS conference consecutively. 

I have attended the SDS'17 (Valencia, Spain), SDS'16 (Berlin, Germany), and SDS'15 (Tempe, AZ, USA) before. I have only missed the first installment of SDS (SDS'14), as I was doing my MSc those days, and my topic was not related to SDS. SDS started as a workshop co-located with IC2E in 2014, and promoted as a symposium in 2016. It graduated as a conference on its own in 2017.

Software-Defined Data Services: Interoperable and Network-Aware Big Data Executions from Pradeeban Kathiravelu

Abstract: Services that access or process a large volume of data are known as data services. Big data frameworks consist of diverse storage media and heterogeneous data formats. Through their service-based approach, data services offer a standardized execution model to big data frameworks. Software-Defined Networking (SDN) increases the programmability of the network, by unifying the control plane centrally, away from the distributed data plane devices. In this paper, we present Software-Defined Data Services (SDDS), extending the data services with the SDN paradigm. SDDS consists of two aspects. First, it models the big data executions as data services or big services composed of several data services. Then, it orchestrates the services centrally in an interoperable manner, by logically separating the executions from the storage. We present the design of an SDDS orchestration framework for network-aware big data executions in data centers. We then evaluate the performance of SDDS through microbenchmarks on a prototype implementation. By extending SDN beyond data centers, we can deploy SDDS in broader execution environments.


This paper received the Best Paper Award at the SDS 2018!

Please read more about this news at the INESC-ID web site.

Is freelancer.com legit or scam?

It is 5.5 years since I reviewed Freelancer.com in my blog post "Is Freelancer.com a scam?". The post remained my most viewed blog post with 200+ comments for quite a long time. Most of the comments are from the users who agree that Freelancer.com is indeed a scam site. I can understand that they found my blog post by doing a quick Google search on "Freelancer.com scam." However, what is more interesting, some readers actually defended Freelancer.com. Interestingly, some of them did not really look genuine (why would you search and land on a blog post discussing whether freelancer.com is a scam unless you were searching for it?) But that it is just a guess. I also noticed canned comments from Freelancer.com employees (most likely bots), suggesting the concerned users to email their support team.

There have been alternatives popping up every once in a while. For example, the HubStaff. However, I am not really comfortable with recommending any of these. As in the past, the ones that many of us suggested as better alternatives to Freelancer.com were later acquired by Freelancer.com anyway. So the new ones can come to the same fate, or they themselves can turn into large evil corporations.

Apart from the issues of landing with a scam employer or a scam employee, Freelancer.com itself is a heavily unethical platform. By searching around the Internet (for example, read Freelancer.com is destroying my life), or reading the tens of comments in my previous blog post on Freelancer.com, you can get a taste of it. Most of the blame on the platform comes from its unethical way of charging. One particular case even I encountered was, the employee is charged the Freelancer.com 10% fees (which can be as high as 100$, if the project is 1000$) even if the employer turns out to be just a scammer. Since the employee did not get any benefit of the platform, any ethical organization would just refund the money. Other obvious issues are lower transaction rates, even lower than your banks, and the painfully long time they take to transfer your money! Transferwise does it at least 10 times faster! That is one ethical company. Freelancer.com on the other hand, an unethical bullying company. The most inhumane characteristics of Freelancer.com is when they randomly choose to block your account and ask for random verifications (as reported by commenters, sometimes driver's licenses are not accepted despite being listed as an accepted form of verification in the Freelancer.com itself).

If you must use sites such as Freelancer.com, try to work from a cheaper third-world country such as Bangladesh or Thailand. Not from Liechtenstein. Because you are not going to get paid higher just because you live in a more expensive or more developed country. 

Moreover, you real-world credentials do not really matter in the Internet world. It is hard to verify on the Internet whether you are who you claim to be. So even if you are an expert, you still have to build your online empire from ground-zero. Start this when you do not really need money, as a past-time. Working on Freelancing websites is not going to give you money quick unless you already have an account with justified success. The early days will be plainly painful as you put in work to a negligible payment (since your competitors are from cheaper countries anyway). 

There are of course many success stories. Many were just lucky or from the third-world countries with a competitive advantage on such global platforms. For example, 5$/hour is high in many South or South East Asian countries, while it is very little for most of the western countries. Others had the patience and time to build their profile slowly, till they can reach the 40 - 50$ hourly rate.

My verdict: Find a real job in the real world if possible. Want to be self-employed? Find local employers and build up a brand, so that you can find clients without greedy middlemen such as Freelancer.com. Freelancer.com is not entirely a scam. But it operates in a gray area with shady business practice (non-existing customer service paired with questionable antics to extract more money from the clients/users), acquiring competitors aiming to operate in a near-monopoly.