Monday, January 28, 2019

Install Kubeflow on Centos 7

This post summarizes installation of Kubeflow on Centos 7, together with its dependencies.

1) Install and enable the COPR Plugin:
$ sudo yum install yum-plugin-copr

$ sudo yum copr enable ngompa/snapcore-el7
Loaded plugins: copr, fastestmirror, langpacks

You are about to enable a Copr repository. Please note that this
repository is not part of the main Fedora distribution, and quality
may vary.

The Fedora Project does not exercise any power over the contents of
this repository beyond the rules outlined in the Copr FAQ at
, and
packages are not held to any quality or securty level.

Please do not file bug reports about these packages in Fedora
Bugzilla. In case of problems, contact the owner of this repository.

Do you want to continue? [y/N]: Y
copr done

2) Install and enable Snapd:
$ sudo yum -y install snapd



$ sudo systemctl enable --now snapd.socket
Created symlink from /etc/systemd/system/sockets.target.wants/snapd.socket to /usr/lib/systemd/system/snapd.socket.

$ sudo ln -s /var/lib/snapd/snap /snap



Now snap is installed. You can confirm it by,
$ snap --help
The snap command lets you install, configure, refresh and remove snaps.
Snaps are packages that work across many different Linux distributions,
enabling secure delivery and operation of the latest apps and utilities.

Usage: snap [...]

Commands can be classified as follows:

         Basics: find, info, install, list, remove
        ...more: refresh, revert, switch, disable, enable
        History: changes, tasks, abort, watch
        Daemons: services, start, stop, restart, logs
       Commands: alias, aliases, unalias, prefer
  Configuration: get, set, wait
        Account: login, logout, whoami
    Permissions: interfaces, interface, connect, disconnect
          Other: version, warnings, okay
    Development: run, pack, try, ack, known, download

For more information about a command, run 'snap help '.
For a short summary of all commands, run 'snap help --all'. 


Install and configure jq:
$ sudo yum install jq -y

$ sudo cat /var/lib/snapd/state.json | jq 'delpaths([["data", "auth", "device"]])' > state.json-new


3) Install MicroK8s:

$ sudo snap install microk8s --classic

2019-01-28T13:46:06-05:00 INFO Waiting for restart...
microk8s v1.13.2 from Canonical✓ installed


Check the currently published versions:
$ snap info microk8s

name:      microk8s
summary:   Kubernetes for workstations and appliances
publisher: Canonical✓
contact:   https://github.com/ubuntu/microk8s
license:   unset
description: |
  MicroK8s is a small, fast, secure, single node Kubernetes that installs on just about any Linux
  box. Use it for offline development, prototyping, testing, or use it on a VM as a small, cheap,
  reliable k8s for CI/CD. It's also a great k8s for appliances - develop your IoT apps for k8s and
  deploy them to MicroK8s on your boxes.
commands:
  - microk8s.config
  - microk8s.disable
  - microk8s.docker
  - microk8s.enable
  - microk8s.inspect
  - microk8s.istioctl
  - microk8s.kubectl
  - microk8s.reset
  - microk8s.start
  - microk8s.status
  - microk8s.stop
services:
  microk8s.daemon-apiserver:          simple, enabled, active
  microk8s.daemon-apiserver-kicker:   simple, enabled, active
  microk8s.daemon-controller-manager: simple, enabled, active
  microk8s.daemon-docker:             simple, enabled, active
  microk8s.daemon-etcd:               simple, enabled, active
  microk8s.daemon-kubelet:            simple, enabled, active
  microk8s.daemon-proxy:              simple, enabled, active
  microk8s.daemon-scheduler:          simple, enabled, active
snap-id:      EaXqgt1lyCaxKaQCU349mlodBkDCXRcg
tracking:     stable
refresh-date: today at 13:46 EST
channels:                             
  stable:         v1.13.2  (383) 229MB classic
  candidate:      v1.13.2  (383) 229MB classic
  beta:           v1.13.2  (383) 229MB classic
  edge:           v1.13.2  (383) 229MB classic
  1.13/stable:    v1.13.2  (384) 229MB classic
  1.13/candidate: v1.13.2  (384) 229MB classic
  1.13/beta:      v1.13.2  (384) 229MB classic
  1.13/edge:      v1.13.2  (384) 229MB classic
  1.12/stable:    v1.12.4  (362) 251MB classic
  1.12/candidate: v1.12.4  (362) 251MB classic
  1.12/beta:      v1.12.4  (362) 251MB classic
  1.12/edge:      v1.12.5  (387) 251MB classic
  1.11/stable:    v1.11.6  (361) 245MB classic
  1.11/candidate: v1.11.6  (361) 245MB classic
  1.11/beta:      v1.11.6  (361) 245MB classic
  1.11/edge:      v1.11.7  (389) 245MB classic
  1.10/stable:    v1.10.12 (364) 200MB classic
  1.10/candidate: v1.10.12 (364) 200MB classic
  1.10/beta:      v1.10.12 (364) 200MB classic
  1.10/edge:      v1.10.12 (364) 200MB classic
installed:        v1.13.2  (383) 229MB classic



More information - https://microk8s.io/#quick-start



4) Install Kubectl:
$ sudo su
# vim /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg


# yum install -y kubectl


$ kubectl cluster-info
Kubernetes master is running at http://localhost:8080

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'

Enable Autocompletion:
$ sudo yum install bash-completion -y
$ echo "source <(kubectl completion bash)" >> ~/.bashrc


5) Install Ksonnet:
$ wget https://github.com/ksonnet/ksonnet/releases/download/v0.13.1/ks_0.13.1_linux_amd64.tar.gz
$ tar -xzvf ks_0.13.1_linux_amd64.tar.gz

Include the location to bashrc.
$ vim ~/.bashrc

export PATH=/home/pkathi2/ks_0.13.1_linux_amd64:$PATH

Open a new terminal and confirm that ks is installed.


$ ks --help

You can use the `ks` commands to write, share, and deploy your Kubernetes
application configuration to remote clusters.

----

Usage:
  ks [flags]
  ks [command]

Available Commands:
  apply       Apply local Kubernetes manifests (components) to remote clusters
  component   Manage ksonnet components
  delete      Remove component-specified Kubernetes resources from remote clusters
  diff        Compare manifests, based on environment or location (local or remote)
  env         Manage ksonnet environments
  generate    Use the specified prototype to generate a component manifest
  help        Help about any command
  import      Import manifest
  init        Initialize a ksonnet application
  module      Manage ksonnet modules
  param       Manage ksonnet parameters for components and environments
  pkg         Manage packages and dependencies for the current ksonnet application
  prototype   Instantiate, inspect, and get examples for ksonnet prototypes
  registry    Manage registries for current project
  show        Show expanded manifests for a specific environment.
  upgrade     Upgrade ks configuration
  validate    Check generated component manifests against the server's API
  version     Print version information for this ksonnet binary

Flags:
      --dir string        Ksonnet application root to use; Defaults to CWD (default "/home/pradeeban/programs/ks_0.13.1_linux_amd64")
  -h, --help              help for ks
      --tls-skip-verify   Skip verification of TLS server certificates
  -v, --verbose count     Increase verbosity. May be given multiple times.

Use "ks [command] --help" for more information about a command.



6) Install Kubeflow:



Add the environment variables to the bashrc.
$ vim ~/.bashrc


export KUBEFLOW_TAG=v0.4.1
export KUBEFLOW_SOURCE=/home/pkathi2/kubeflow_src
export KFAPP=kfapp





$ mkdir ${KUBEFLOW_SOURCE}
$ cd ${KUBEFLOW_SOURCE}

$ curl https://raw.githubusercontent.com/kubeflow/kubeflow/${KUBEFLOW_TAG}/scripts/download.sh | bash





$ scripts/kfctl.sh init ${KFAPP} --platform none
$ cd ${KFAPP}

Create Context:

Follow the instructions here.
$ kubectl config --kubeconfig=$HOME/.kube/sa-config set-context svcs-acct-context

Context "svcs-acct-context" created.


$ ${
KUBEFLOW_SOURCE}/scripts/kfctl.sh generate k8s
$ ${
KUBEFLOW_SOURCE}/scripts/kfctl.sh apply k8s

Thursday, January 17, 2019

Fake reviews and Amazon.com

I used to be an active reviewer in TripAdvisor, until one day. That day, as usual, I was going through TripAdvisor, reviewing places I have been before. I found a large segment of fake reviews. I reported them to TripAdvisor with proof. They did nothing. In fact, an average Indian restaurant had become the #1 in Lisboa, thanks to fake reviews! TripAdvisor failed to take action even though I proved it with substantial evidence. Of course, eventually more people visited, and real reviews started coming in more than the fake ones in number. TripAdvisor re-adjusted the rating to #1,532 of 4,361 Restaurants in Lisbon, from its previous spot, when I checked now (2 years later). [Read the full story]

Rating, once fake reviews are removed
This time I fell for fake reviews on Amazon. A product that does not even work properly got all 5* reviews, thanks to fakes. As of now, with 27 reviews, it is 4.5*, including my 1*. When I bought it, it was all 5* reviews! Then, today I found an excellent website named fakespot.com that identifies fake reviews. It recognizes and eleminates the fake reviews and gives you the correct review back. So according to that site, this product deserves just 1.5*. Indeed, an accurate rating! There is also another website called reviewmeta.com which does not seem to work as good as fakespot.com.
WSJ has made an excellent video on these fake reviews. You should watch it.


Another site I like to use often is camelcamelcamel.com, which identifies whether a discount is really a discount by tracing the pricing variations for the products sold on Amazon.

Make sure to check the reviews of the other products from the same vendor when purchasing something. Currently, the product in question has already been sold out, and I am not even sure whether the seller will sell it again. Probably they bought some of these in Chinese street market and sold them all successfully soon after. Therefore, blacklisting or making aware of the buyers of a single product is not going to work. This has to happen at making aware of shady vendors, not just their dubious products. Next time, need to be a bit more vigilant when shopping in Amazon. The fakes are improving their game.


Update (Jan 19th):
Now with my 1-* review together with someone else's 1-* review (which I think the only honest reviews) in consideration, Fakespot has updated its rating to the product as 0-*. Indeed the ideal rating for this product. I wish Amazon lets me give 0-* ratings. :) 

  • How are reviewers describing this item?
    good, easy, nice, little and better.
  • Our engine has profiled the reviewer patterns and has determined that there is high deception involved.
  • Our engine has analyzed and discovered that 16.1% of the reviews are reliable.
  • This product had a total of 31 reviews on Jan 19 2019.
 

Saturday, January 5, 2019

Restaurant Scams in Atlanta - II

So this gratuity scam happened to us once more in Atlanta. This time it was Terrapin Taproom in Battery Atlanta. They overcharged us by 4.50$. The waitress must have increased the gratuity value we entered. I called to complain to the restaurant. They said their manager would call me back. I am waiting for her call now. I will update this post once I heard back from them.

Previously Doc Chey's Dragon Bowl overcharged me by 1$. I ignored it since it is too little of value to waste my time and they did not give me the receipt either. I should have asked. But nevermind.

The first time was from an Indian restaurant, which charged us 6$ extra. What can be worse? There was a 20% mandatory service fee (since we were a group of 6), and with the 6$ extra that they overcharged me, it made a 40% tip. But I complained to my bank and got it refunded. If you are interested, please go ahead and read the full story.

It seems this is a common scam in Atlanta. We have encountered this three times so far within less than a year. This scam never happened to us in other ~100 cities during our nomadic life of past six years.

I found from the news that waiters padding tip amounts on credit/debit card payments are not entirely new, or specific to Atlanta. It is going on quite some time in the US, apparently. One waiter even got arrested for this. There is also a question on Quora, discussing this same issue.

If you are a tourist or someone who wants to dine out in Atlanta, make sure to check your bank transactions afterward if you pay by card. Or better, just pay by cash. It is safer to pay by cash if you are a tourist from a foreign country since it will be harder for you to go after them.

If you are a waiter reading this post, please for the sake of God, stop altering the value that we entered as the tip. It is shady and a scam, and you cannot justify it by any means. Don't assume customers are made of cash. Many of us are students or in a tough economic situation. We cannot afford to lose money to scams just because we eat out now and then.


Update: Terrapin did an internal investigation and reprimanded the waitress who scammed the customers. They called me to inform me that this event would not repeat in the future and confirmed that they would refund the overcharge (4.50$) to me and any others (if any others found during their further investigation). I am happy that they made an honest effort to rectify this problem.

Friday, December 28, 2018

A few things that made my 2018 interesting..

A day in Monaco
2018 is one of my most favorite years. It gave me lots of happy moments. Adhering to the habit I started in 2012, this post summarizes 30 things that made my 2018 interesting.

1. Submitting my Ph.D. thesis
I planned to defend by 2018 October. But it got delayed, and I will defend early next year. However, I have submitted the thesis, and my Ph.D. research is complete. :)

2. Return to Emory University, Atlanta
It is nice to be back at Emory BMI for a long-term after my previous short stint in 2016. Everything felt the same and different at the same time.

3. A day in Monaco, walking across the entire country
It is trivial to walk across a country if it is as small as Monaco. :)

4. The conclusion of Ph.D. collaborations
It is important to conclude the collaborations on a positive note. As my Ph.D. nears the end, I am happy that my collaborations from URijeka (SDSC) and KAUST (NetUber and Dynam-IX) also came to successful completion.

5. Sleepless nights in Shenzhen, China
When deadlines come, you get to work wherever you are. I ended up working overnights for the successful submission of Óbidos.

6. Shopping in rainy and windy days in Jeju
I was lucky (or was that Jeju, which indeed was fortunate to receive me?). After I left Jeju, in a few months, I learned that Jeju does not offer visa-free entry to Sri Lankans anymore.

7. Best Paper Award at SDS'18
I received the best paper award at SDS'18 for my paper on Software-Defined Data Services (SDDS). It was an exciting moment.

Barcelona, once again!
8. Random Walks in Barcelona, once more!
This is a city I adore. It was nice to be back yet one more time. I think I am getting familiar with this city more and more.

9. Cruise in Zurich Lake
Zurich was still cold in May. Of course not as cold as my last visit for the 2017 New Year's Eve. But still raining.

10. IFIP Networking 2018 with an overdose of caffeine
IFIP is a conference I always wanted to participate. Had the chance to present NetUber there this year, finally.

11. Spending a whole day in Sesimbra Beach
Sesimbra is a beach town close to Lisboa in the Setúbal district. I always wanted to go there. I got the chance finally this year.

12. First journal papers and book chapters of my Ph.D.
I started working on 3 journal papers (Évora, Óbidos, and SD-CPS) and 2 book chapters (SDN for big data and SDN for service compositions) last year. They all got accepted and published this year!

13. The views of French Riviera
Despite the predicted weather that it would be warmer than Barcelona, Nice turned out to be cloudy and colder.

14. Fresh Seafood in Jeju
Watching the show of octopus, squids, and several similar sea creatures being cooked alive for me.

15. Leaving Portugal and EU after 6 years
I don't think I will ever forget those sweet memories of Lisboa. Experiencing 'saudade', as we call it in Portugal.

16. Almada, just across the Tagus River.
Pleasant environment, but busy days.

17. Presenting my CAT at IST
I should have tried to make the IST's CAT and UCL's confirmation as a single presentation. But due to the difference in UCL and IST requirements, I chose to make 2 different presentations. With the CAT and my thesis submission, I have met all the requirements for the thesis (ABD/All But Dissertation).

18. A slow-paced year, quite the opposite of 2017
2017 was way too fast with several migrations and travels. 2018, on the other hand, was more stable and slow-paced. This was the first time during the past 6 years that we stayed home during the new year's eve. Feeling somewhat old.

19. Becoming a morning person in Atlanta
Love those early morning commutes to work while it is still dark and cold outside.

20. Hopping on to random buses in Jeju, hoping to reach the destination eventually.
Rainy days in Jeju
And we did. I even made an algorithm out of this.

21. Bus Commutes from Calçada de Carriche to my lab

My apartment was in Calçada de Carriche, a bit farther from my lab, INESC-ID Lisboa, compared to my previous neighborhoods. Therefore, needed a relatively longer bus ride.

22. Parque das Nações revamped.
Parque das Nações is my favorite spot in Lisboa. It has improved within the past few years with new restaurants, yet maintained its charm. It felt nostalgic to be back there.

23. Exploring more of greater Lisboa
Even after 6 years in Lisboa, we realized that there were a few places that we hadn't visited. This year, we visited the Loures region and its big shopping mall for the first time. 

24. Évora after 4 years
Évora has its historical significance. It is also where I was informed that I was selected to the prestigious EMJD-DC program with the grant, 4 years back.

25. Trying to relive moments in Portugal
As we count the last days in Lisboa, we tried our best to visit our favorite places just one more time, with mixed success.
The entire botanical garden was shining in colorful lights and music during the December nights.

27. Ponce City Market, Atlanta
I like the interiors of this small-scale shopping mall, especially its food court. It gives strong memories.

28. Assembling Furniture from IKEA
IKEA furniture shopping and assembling them for the first time. A fun experience.

29. Atlanta Christkindl Market
Christmas markets are a significant part of the festival season. This market in Atlanta reminded me of the ones I experienced in EU during the past 6 years, such as, Lisbon, Stockholm, Helsinki, and Louvain-la-Neuve Christmas markets.

30. Poble Espanyol, Barcelona
An open-air museum that summarizes entire Spain. Not too big. But it was a pleasant walk with architectures and some snacks from different Spanish regions.



Every year, I have one new year's resolution - to outperform my previous year. :) I already have high expectations for 2019. I wish you a happy new year everyone. Thanks for reading my list until the end. You may also read the blog posts of all the previous years (2018 - 2012) as well.

Thursday, November 29, 2018

Add additional context to SELinux

In this post we quickly look into how we add context for SELinux on our Orthanc binary storage.
$ sudo semanage fcontext -a -t etc_t -s system_u /opt/localdrive/orthancstorage

$ sudo restorecon -R -v /opt/localdrive/orthancstorage

Now we confirm it by the below command:
$ cat /etc/selinux/targeted/contexts/files/file_contexts.local
# This file is auto-generated by libsemanage
# Do not edit directly.

/opt/localdrive/postgres(/.*)?    system_u:object_r:postgresql_db_t:s0
/opt/localdrive/orthancstorage    system_u:object_r:etc_t:s0


Confirm the updates
$ ls -laZ /opt/localdrive/orthancstorage