Monday, January 28, 2019

Install Kubeflow on Centos 7

This post summarizes installation of Kubeflow on Centos 7, together with its dependencies.

1) Install and enable the COPR Plugin:
$ sudo yum install yum-plugin-copr

$ sudo yum copr enable ngompa/snapcore-el7
Loaded plugins: copr, fastestmirror, langpacks

You are about to enable a Copr repository. Please note that this
repository is not part of the main Fedora distribution, and quality
may vary.

The Fedora Project does not exercise any power over the contents of
this repository beyond the rules outlined in the Copr FAQ at
, and
packages are not held to any quality or securty level.

Please do not file bug reports about these packages in Fedora
Bugzilla. In case of problems, contact the owner of this repository.

Do you want to continue? [y/N]: Y
copr done

2) Install and enable Snapd:
$ sudo yum -y install snapd



$ sudo systemctl enable --now snapd.socket
Created symlink from /etc/systemd/system/sockets.target.wants/snapd.socket to /usr/lib/systemd/system/snapd.socket.

$ sudo ln -s /var/lib/snapd/snap /snap



Now snap is installed. You can confirm it by,
$ snap --help
The snap command lets you install, configure, refresh and remove snaps.
Snaps are packages that work across many different Linux distributions,
enabling secure delivery and operation of the latest apps and utilities.

Usage: snap [...]

Commands can be classified as follows:

         Basics: find, info, install, list, remove
        ...more: refresh, revert, switch, disable, enable
        History: changes, tasks, abort, watch
        Daemons: services, start, stop, restart, logs
       Commands: alias, aliases, unalias, prefer
  Configuration: get, set, wait
        Account: login, logout, whoami
    Permissions: interfaces, interface, connect, disconnect
          Other: version, warnings, okay
    Development: run, pack, try, ack, known, download

For more information about a command, run 'snap help '.
For a short summary of all commands, run 'snap help --all'. 


Install and configure jq:
$ sudo yum install jq -y

$ sudo cat /var/lib/snapd/state.json | jq 'delpaths([["data", "auth", "device"]])' > state.json-new


3) Install MicroK8s:

$ sudo snap install microk8s --classic

2019-01-28T13:46:06-05:00 INFO Waiting for restart...
microk8s v1.13.2 from Canonical✓ installed


Check the currently published versions:
$ snap info microk8s

name:      microk8s
summary:   Kubernetes for workstations and appliances
publisher: Canonical✓
contact:   https://github.com/ubuntu/microk8s
license:   unset
description: |
  MicroK8s is a small, fast, secure, single node Kubernetes that installs on just about any Linux
  box. Use it for offline development, prototyping, testing, or use it on a VM as a small, cheap,
  reliable k8s for CI/CD. It's also a great k8s for appliances - develop your IoT apps for k8s and
  deploy them to MicroK8s on your boxes.
commands:
  - microk8s.config
  - microk8s.disable
  - microk8s.docker
  - microk8s.enable
  - microk8s.inspect
  - microk8s.istioctl
  - microk8s.kubectl
  - microk8s.reset
  - microk8s.start
  - microk8s.status
  - microk8s.stop
services:
  microk8s.daemon-apiserver:          simple, enabled, active
  microk8s.daemon-apiserver-kicker:   simple, enabled, active
  microk8s.daemon-controller-manager: simple, enabled, active
  microk8s.daemon-docker:             simple, enabled, active
  microk8s.daemon-etcd:               simple, enabled, active
  microk8s.daemon-kubelet:            simple, enabled, active
  microk8s.daemon-proxy:              simple, enabled, active
  microk8s.daemon-scheduler:          simple, enabled, active
snap-id:      EaXqgt1lyCaxKaQCU349mlodBkDCXRcg
tracking:     stable
refresh-date: today at 13:46 EST
channels:                             
  stable:         v1.13.2  (383) 229MB classic
  candidate:      v1.13.2  (383) 229MB classic
  beta:           v1.13.2  (383) 229MB classic
  edge:           v1.13.2  (383) 229MB classic
  1.13/stable:    v1.13.2  (384) 229MB classic
  1.13/candidate: v1.13.2  (384) 229MB classic
  1.13/beta:      v1.13.2  (384) 229MB classic
  1.13/edge:      v1.13.2  (384) 229MB classic
  1.12/stable:    v1.12.4  (362) 251MB classic
  1.12/candidate: v1.12.4  (362) 251MB classic
  1.12/beta:      v1.12.4  (362) 251MB classic
  1.12/edge:      v1.12.5  (387) 251MB classic
  1.11/stable:    v1.11.6  (361) 245MB classic
  1.11/candidate: v1.11.6  (361) 245MB classic
  1.11/beta:      v1.11.6  (361) 245MB classic
  1.11/edge:      v1.11.7  (389) 245MB classic
  1.10/stable:    v1.10.12 (364) 200MB classic
  1.10/candidate: v1.10.12 (364) 200MB classic
  1.10/beta:      v1.10.12 (364) 200MB classic
  1.10/edge:      v1.10.12 (364) 200MB classic
installed:        v1.13.2  (383) 229MB classic



More information - https://microk8s.io/#quick-start



4) Install Kubectl:
$ sudo su
# vim /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg


# yum install -y kubectl


$ kubectl cluster-info
Kubernetes master is running at http://localhost:8080

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'

Enable Autocompletion:
$ sudo yum install bash-completion -y
$ echo "source <(kubectl completion bash)" >> ~/.bashrc


5) Install Ksonnet:
$ wget https://github.com/ksonnet/ksonnet/releases/download/v0.13.1/ks_0.13.1_linux_amd64.tar.gz
$ tar -xzvf ks_0.13.1_linux_amd64.tar.gz

Include the location to bashrc.
$ vim ~/.bashrc

export PATH=/home/pkathi2/ks_0.13.1_linux_amd64:$PATH

Open a new terminal and confirm that ks is installed.


$ ks --help

You can use the `ks` commands to write, share, and deploy your Kubernetes
application configuration to remote clusters.

----

Usage:
  ks [flags]
  ks [command]

Available Commands:
  apply       Apply local Kubernetes manifests (components) to remote clusters
  component   Manage ksonnet components
  delete      Remove component-specified Kubernetes resources from remote clusters
  diff        Compare manifests, based on environment or location (local or remote)
  env         Manage ksonnet environments
  generate    Use the specified prototype to generate a component manifest
  help        Help about any command
  import      Import manifest
  init        Initialize a ksonnet application
  module      Manage ksonnet modules
  param       Manage ksonnet parameters for components and environments
  pkg         Manage packages and dependencies for the current ksonnet application
  prototype   Instantiate, inspect, and get examples for ksonnet prototypes
  registry    Manage registries for current project
  show        Show expanded manifests for a specific environment.
  upgrade     Upgrade ks configuration
  validate    Check generated component manifests against the server's API
  version     Print version information for this ksonnet binary

Flags:
      --dir string        Ksonnet application root to use; Defaults to CWD (default "/home/pradeeban/programs/ks_0.13.1_linux_amd64")
  -h, --help              help for ks
      --tls-skip-verify   Skip verification of TLS server certificates
  -v, --verbose count     Increase verbosity. May be given multiple times.

Use "ks [command] --help" for more information about a command.



6) Install Kubeflow:



Add the environment variables to the bashrc.
$ vim ~/.bashrc


export KUBEFLOW_TAG=v0.4.1
export KUBEFLOW_SOURCE=/home/pkathi2/kubeflow_src
export KFAPP=kfapp





$ mkdir ${KUBEFLOW_SOURCE}
$ cd ${KUBEFLOW_SOURCE}

$ curl https://raw.githubusercontent.com/kubeflow/kubeflow/${KUBEFLOW_TAG}/scripts/download.sh | bash





$ scripts/kfctl.sh init ${KFAPP} --platform none
$ cd ${KFAPP}

Create Context:

Follow the instructions here.
$ kubectl config --kubeconfig=$HOME/.kube/sa-config set-context svcs-acct-context

Context "svcs-acct-context" created.


$ ${
KUBEFLOW_SOURCE}/scripts/kfctl.sh generate k8s
$ ${
KUBEFLOW_SOURCE}/scripts/kfctl.sh apply k8s


To uninstall:

Remove the entries from bashrc appropriately.


$ sudo yum remove kubectl

$ sudo snap remove microk8s

Thursday, January 17, 2019

Fake reviews and Amazon.com

I used to be an active reviewer in TripAdvisor, until one day. That day, as usual, I was going through TripAdvisor, reviewing places I have been before. I found a large segment of fake reviews. I reported them to TripAdvisor with proof. They did nothing. In fact, an average Indian restaurant had become the #1 in Lisboa, thanks to fake reviews! TripAdvisor failed to take action even though I proved it with substantial evidence. Of course, eventually more people visited, and real reviews started coming in more than the fake ones in number. TripAdvisor re-adjusted the rating to #1,532 of 4,361 Restaurants in Lisbon, from its previous spot, when I checked now (2 years later). [Read the full story]

Rating, once fake reviews are removed
This time I fell for fake reviews on Amazon. A product that does not even work properly got all 5* reviews, thanks to fakes. As of now, with 27 reviews, it is 4.5*, including my 1*. When I bought it, it was all 5* reviews! Then, today I found an excellent website named fakespot.com that identifies fake reviews. It recognizes and eleminates the fake reviews and gives you the correct review back. So according to that site, this product deserves just 1.5*. Indeed, an accurate rating! There is also another website called reviewmeta.com which does not seem to work as good as fakespot.com.
WSJ has made an excellent video on these fake reviews. You should watch it.


Another site I like to use often is camelcamelcamel.com, which identifies whether a discount is really a discount by tracing the pricing variations for the products sold on Amazon.

Make sure to check the reviews of the other products from the same vendor when purchasing something. Currently, the product in question has already been sold out, and I am not even sure whether the seller will sell it again. Probably they bought some of these in Chinese street market and sold them all successfully soon after. Therefore, blacklisting or making aware of the buyers of a single product is not going to work. This has to happen at making aware of shady vendors, not just their dubious products. Next time, need to be a bit more vigilant when shopping in Amazon. The fakes are improving their game.


Update (Jan 19th):
Now with my 1-* review together with someone else's 1-* review (which I think the only honest reviews) in consideration, Fakespot has updated its rating to the product as 0-*. Indeed the ideal rating for this product. I wish Amazon lets me give 0-* ratings. :) 

  • How are reviewers describing this item?
    good, easy, nice, little and better.
  • Our engine has profiled the reviewer patterns and has determined that there is high deception involved.
  • Our engine has analyzed and discovered that 16.1% of the reviews are reliable.
  • This product had a total of 31 reviews on Jan 19 2019.

Update (March 17th):
The seller contacted me over the phone (from China!) and email and tried to negotiate me to delete the review by offering me increasing offers of 10$, 30$, and 50$ with a full refund with the product as free. I already returned the product and got the full refund during the refund window. I sent their email communications with all the proofs to Amazon and asked them to take action. Amazon failed to take action and the seller managed to delete some (at least one 1-star review, I noticed) negative reviews with similar bribes.



Their email goes like this:


***************************************************

Regarding your Amazon Product Review


Dear Customer,
Thank you for purchasing our Active Stylus Pen
Thank you for your purchase and taking the time to write a product review. We are terribly sorry to hear the product you received is defective and would like to know if we can send you a free replacement or assist you with a refund.

Customer reviews is important to us and we value your response. All responses will be used to further improve the quality of our service and products.

We saw the 1 star review you wrote down on January 16. This has a great impact on us. We are just a small seller. This will cause great harm to us. I want to ask you to help me delete this 1 star review. Can you help me?

We can provide you with a full refund and the product will be given as a gift, and we will pay you an additional $50 as compensation.

Can you accept it?

Sorry for the inconvenience and thank you for giving us the opportunity to rectify the matter.

Looking forward to your reply

Sincerly yours
Milletech Customer Service Team





***************************************************




Of course, my review will remain there. I truly wish Amazon was more proactive in removing or restricting these fake sellers. The product is thriving with more and more fake reviews. Amazon took 0 action to protect the customers from this shady seller. On the other hand, this is just a sample. I am sure that this is not the first or the last product to use fake reviews to boost the sales. Amazon agreed that the seller got my phone number and email address from their system. It is really bad that Amazon discloses this information to the seller without a valid reason. This information enables the sellers to bribe or threaten the customers to leave positive reviews.

Finally, I ask everyone to do additional research rather than blindly trusting the reviews they find online. Remember that a review you read online may have been incentivized even if they do not explicitly say so.

Saturday, January 5, 2019

Restaurant Scams in Atlanta - II

So this gratuity scam happened to us once more in Atlanta. This time it was Terrapin Taproom in Battery Atlanta. They overcharged us by 4.50$. The waitress must have increased the gratuity value we entered. I called to complain to the restaurant. They said their manager would call me back. I am waiting for her call now. I will update this post once I heard back from them.

Previously Doc Chey's Dragon Bowl overcharged me by 1$. I ignored it since it is too little of value to waste my time and they did not give me the receipt either. I should have asked. But nevermind.

The first time was from an Indian restaurant, which charged us 6$ extra. What can be worse? There was a 20% mandatory service fee (since we were a group of 6), and with the 6$ extra that they overcharged me, it made a 40% tip. But I complained to my bank and got it refunded. If you are interested, please go ahead and read the full story.

It seems this is a common scam in Atlanta. We have encountered this three times so far within less than a year. This scam never happened to us in other ~100 cities during our nomadic life of past six years.

I found from the news that waiters padding tip amounts on credit/debit card payments are not entirely new, or specific to Atlanta. It is going on quite some time in the US, apparently. One waiter even got arrested for this. There is also a question on Quora, discussing this same issue.

If you are a tourist or someone who wants to dine out in Atlanta, make sure to check your bank transactions afterward if you pay by card. Or better, just pay by cash. It is safer to pay by cash if you are a tourist from a foreign country since it will be harder for you to go after them.

If you are a waiter reading this post, please for the sake of God, stop altering the value that we entered as the tip. It is shady and a scam, and you cannot justify it by any means. Don't assume customers are made of cash. Many of us are students or in a tough economic situation. We cannot afford to lose money to scams just because we eat out now and then.


Update: Terrapin did an internal investigation and reprimanded the waitress who scammed the customers. They called me to inform me that this event would not repeat in the future and confirmed that they would refund the overcharge (4.50$) to me and any others (if any others found during their further investigation). I am happy that they made an honest effort to rectify this problem.