Sunday, December 15, 2019

Data Services with Bindaas: RESTful Interfaces for Diverse Data Sources

Dec 11th, 2019: Today I presented my first conference paper from my Postdoc at IEEE big data 2019 conference held in Big Data 2019. It was fun listening to various talks and networking with several researchers from all across the world.

Abstract: The diversity of data management systems affords developers the luxury of building heterogeneous architectures to address the unique needs of big data. It allows one to mix-n-match systems that can store, query, update, and process data based on specific use cases. However, this heterogeneity brings with it the burden of developing custom interfaces for each data management system. Existing big data frameworks fall short in mitigating these challenges imposed. In this paper, we present Bindaas, a secure and extensible big data middleware that offers uniform access to diverse data sources. By providing a RESTful web service interface to the data sources, Bindaas exposes query, update, store, and delete functionality of the data sources as data service APIs, while providing turn-key support for standard operations involving access control and audit-trails. The research community has deployed Bindaas in various production environments in healthcare. Our evaluations highlight the efficiency of Bindaas in serving concurrent requests to data source instances with minimal overheads.

Thursday, December 5, 2019

Running Kheops on a Mac OS host

This post discusses how to run Kheops from the on Mac OS, with Docker.

$ git clone git@github.com:OsiriX-Foundation/kheopsDocker.git

$ cd kheopsDocker/

$ sh script_run_docker.sh

Install newman

$ sudo npm install -g newman

/usr/local/bin/newman -> /usr/local/lib/node_modules/newman/bin/newman.js
+ newman@4.5.6
added 160 packages from 196 contributors in 5.481s


   ╭────────────────────────────────────────────────────────────────╮
   │                                                                │
   │       New minor version of npm available! 6.9.0 → 6.13.2       │
   │   Changelog: https://github.com/npm/cli/releases/tag/v6.13.2   │
   │               Run npm install -g npm to update!                │
   │                                                                │
   ╰────────────────────────────────────────────────────────────────╯



Replace line 10 in script_run_docker.sh

with

newman run postman/kheops-Get_tokens.json -e postman/kheops_localhost.postman_environment.json -g postman/Workspace.postman_globals.json --export-globals postman/Workspace.postman_globals.json

Save the file.

If you are using Docker for Mac, you may encounter the error listed in https://github.com/docker/for-mac/issues/2396

Creating pacsldap ... error
Creating db       ...

ERROR: for pacsldap  Cannot start service ldap: b'Mounts denied: \r\nThe paths /etc/localtime and /usr/share/zoneinfo/Europe/Zurich\r\nare not shared from OS X and are not known to Docker.\r\nYou can configure shared paths from Docker -> Preferences... -> File Sharing.\r\Creating db       ... done

ERROR: for ldap  Cannot start service ldap: b'Mounts denied: \r\nThe paths /etc/localtime and /usr/share/zoneinfo/Europe/Zurich\r\nare not shared from OS X and are not known to Docker.\r\nYou can configure shared paths from Docker -> Preferences... -> File Sharing.\r\nSee https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.\r\n.'
ERROR: Encountered errors while bringing up the project.

As a work-around, remove all the occurrences of the below line from kheops/docker-compose.yml.
      - /etc/localtime:/etc/localtime:ro

$ sudo mkdir /dcm4chee-arc

$ sudo chmod 777 /dcm4chee-arc

Share /usr/share/zoneinfo/Europe/ and /dcm4chee-arc from the Preferences of Docker Desktop in your Mac host machine.

Save the file and then re-run

$ sh script_run_docker.sh

$ sh script_run_docker.sh
Stopping db       ... done
Stopping pacsldap ... done
Removing pacsarc  ... done
Removing db       ... done
Removing pacsldap ... done
Removing network kheops_default
Removing volume kheops_dcm4chee-ldap-ldap
Removing volume kheops_dcm4chee-ldap-slapdd
Removing volume kheops_dcm4chee-db-data
Removing volume kheops_dcm4chee-arc-wildfly
Removing volume kheops_kheops-db-data
Creating network "kheops_default" with the default driver
Creating volume "kheops_dcm4chee-ldap-ldap" with default driver
Creating volume "kheops_dcm4chee-ldap-slapdd" with default driver
Creating volume "kheops_dcm4chee-db-data" with default driver
Creating volume "kheops_dcm4chee-arc-wildfly" with default driver
Creating volume "kheops_kheops-db-data" with default driver
Creating db       ... done
Creating pacsldap ... done
Creating pacsarc  ... done
Creating kheopszipper           ... done
Creating kheopsauthorization    ... done
Creating pacsauthorizationproxy ... done
Creating kheopsui               ... done
Creating kheopsdicomwebproxy    ... done
Creating kheopsreverseproxy     ... done
CONTAINER ID        IMAGE                                              COMMAND                  CREATED             STATUS              PORTS                                                                                                                                              NAMES
d5caa14fac54        osirixfoundation/kheops-reverse-proxy:ci-test      "./etc/nginx/conf.d/…"   47 seconds ago      Up 45 seconds       0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:8042->8042/tcp, 0.0.0.0:8048->8048/tcp                                                           kheopsreverseproxy
67552e9c9469        osirixfoundation/kheops-dicomweb-proxy:master      "./replaceSecretsAnd…"   48 seconds ago      Up 46 seconds       0.0.0.0:8049->8080/tcp                                                                                                                             kheopsdicomwebproxy
92710a77dd25        osirixfoundation/kheops-ui:master                  "./docker-entrypoint…"   48 seconds ago      Up 46 seconds       80/tcp, 3000/tcp                                                                                                                                   kheopsui
c9fa33eb693d        osirixfoundation/kheops-authorization:dev          "./replaceSecretsAnd…"   50 seconds ago      Up 47 seconds       8080/tcp                                                                                                                                           kheopsauthorization
0f267500942f        osirixfoundation/kheops-zipper:master              "./replaceSecretsAnd…"   50 seconds ago      Up 48 seconds       8080/tcp                                                                                                                                           kheopszipper
c95c6e4380c6        osirixfoundation/pacs-authorization-proxy:master   "/bin/sh -c '. $NGIN…"   50 seconds ago      Up 48 seconds       0.0.0.0:82->80/tcp                                                                                                                                 pacsauthorizationproxy
89c52e69534a        osirixfoundation/kheops-dcm4chee-arc-psql:latest   "/script_entrypoint.…"   51 seconds ago      Up 49 seconds       0.0.0.0:2575->2575/tcp, 0.0.0.0:8080->8080/tcp, 0.0.0.0:8443->8443/tcp, 0.0.0.0:8787->8787/tcp, 0.0.0.0:9990->9990/tcp, 0.0.0.0:11112->11112/tcp   pacsarc
78bf11251bf7        dcm4che/slapd-dcm4chee:2.4.44-15.0                 "/docker-entrypoint.…"   52 seconds ago      Up 50 seconds       389/tcp                                                                                                                                            pacsldap
f4628a1a1c44        osirixfoundation/kheops-database:demo              "verify-secret-env-v…"   52 seconds ago      Up 50 seconds       5432/tcp                                                                                                                                           db
newman

kheops-Get_tokens

→ well-known
  GET https://keycloak.kheops.online/auth/realms/travis/.well-known/openid-configuration [200 OK, 2.62KB, 657ms]

→ get token user 1 (titi)
  POST https://keycloak.kheops.online/auth/realms/travis/protocol/openid-connect/token [200 OK, 2.01KB, 325ms]

→ get token user 2 (toto)
  POST https://keycloak.kheops.online/auth/realms/travis/protocol/openid-connect/token [200 OK, 2.01KB, 314ms]

→ get token user 3 (tata)
  POST https://keycloak.kheops.online/auth/realms/travis/protocol/openid-connect/token [200 OK, 2.01KB, 332ms]

┌─────────────────────────┬─────────────────────┬────────────────────┐
│                         │            executed │             failed │
├─────────────────────────┼─────────────────────┼────────────────────┤
│              iterations │                   1 │                  0 │
├─────────────────────────┼─────────────────────┼────────────────────┤
│                requests │                   4 │                  0 │
├─────────────────────────┼─────────────────────┼────────────────────┤
│            test-scripts │                   4 │                  0 │
├─────────────────────────┼─────────────────────┼────────────────────┤
│      prerequest-scripts │                   0 │                  0 │
├─────────────────────────┼─────────────────────┼────────────────────┤
│              assertions │                   0 │                  0 │
├─────────────────────────┴─────────────────────┴────────────────────┤
│ total run duration: 1790ms                                         │
├────────────────────────────────────────────────────────────────────┤
│ total data received: 7.53KB (approx)                               │
├────────────────────────────────────────────────────────────────────┤
│ average response time: 407ms [min: 314ms, max: 657ms, s.d.: 144ms] │
└────────────────────────────────────────────────────────────────────┘

Then go to http://localhost:8042 from your favorite browser.

Register yourself when it redirects to a page such as https://keycloak.kheops.online/auth/realms/travis/protocol/openid-connect/auth?client_id=loginConnect&redirect_uri=.........

It will take you back to http://localhost:8042/inbox after the registration. Now Kheops is ready to use!
Kheops Web UI

I can even tweet from the Kheops Web UI with a link to the created album!
"My KHEOPS shared album. http://localhost/view/ttuArLPrFv834JQ88KaQGY #KHEOPS"
Apparently, the link is broken. It should be: http://localhost:8042/view/ttuArLPrFv834JQ88KaQGY
But I see the parallels with the concept of Óbidos (MEDIator) replicasets in the given UUID ttuArLPrFv834JQ88KaQGY.
One can also create a sharing URL with upload/edit/delete capabilities (again, the port is not correctly used. But that is a front-end bug they can fix. For our purposes, we just need to add the port in the shared URL).

I can also view the images that I uploaded from OHIF viewer directly, by clicking that link. It uses their cloud hosted instance of the OHIF viewer to view the image stored locally, as can be seen from the URL.

Monday, December 2, 2019

Those who ruin the flights for others

Batu Caves Murugan
The last week I had a long return flight to Kuala Lumpur from Atlanta, via Doha. Usually, I watch a lot of movies during the flights. But this time, I decided to watch just 2 movies during the entire journey (to and from).

In any economy class, there are 3 (or 4, if you sit in a middle seat) people who can ruin your flight. First, the person who sits in the seat behind you unless you sit in the last row. Second, the person who sits in front of you, unless you sit in the exit row or the first row in the block. Finally, the person(s) who sit(s) next to you.

The person who sits behind you has the potential to ruin your journey entirely. They can kick your seat non-stop. Some people do not realize how weak the economy class seat is. If you put your feet on the seat tray, or if you just constantly jerk it, the movements entirely affect the seat in front of you. It gives headaches. Please don't do that. Infants have the habit of kicking. If possible, make sure their feet are not directed towards the seat in front of them.

Second, the person who sits in front of you. They may recline their seat any time, giving your knee a sudden jerk. Some idiots choose to do that while the meals are served, making you spill your food or drink on you. I usually avoid reclining my seat (except maybe by a negligible degree). One exception: if you are in the seat behind me and continue to kick or jerk my seat, I WILL recline my seat to the fullest.

Finally, the person who sits next to you. They may occupy your seat area, lean on you and sleep, or cross their leg and touch you with their shoe or smelly socks. There was a time I was sandwiched in a middle seat with people in either side (window and aisle seats) occupying both handrests. People, please give that armrest to the middleseater! On the other hand, once in a flight, as always, I let the middle-seat person use the entire armrest. But after a few minutes, I noticed I was being pushed to a corner. Then I found that that person has folded the armrest, so that they can invade my space! Of course, I agree that they are a big person. However, it is NOT okay. I instantly put the armrest down back in its place, effectively cutting the huge person's belly which was invading my space by 25%. The person realized their mistake and retracted.

I wish everyone tries to be mindful of the surroundings and the neighbours. This will make our economy class flight experience much better.

Sunday, November 17, 2019

Revising Portuguese with Duolingo

Since I successfully defended my PhD thesis this year and left Portugal, I started to miss Portugal and hearing Portuguese. I also do not want to forget Portuguese, the language of the country that I love most. Therefore, I started practicing Portuguese with Duolingo 10 days ago. So far, good progress. Duolingo is funny with its interesting lessons and stories.

Some interesting events from Duolingo.

1. Luis is single-handedly responsible for the pineapple shortage in the world.
Image 


2.  and this woman who can eat eighteen! sandwiches.
Image 
3.  Proud to live in a world where even the cats read books.
Image 
4. and it gets creepier..
Image


5. Tell me about these unfair expectations..
Image 
6. A vegan tiger.

7. The mythical armadillo strikes again.. :D
Well, duolingo has a thing with this tatu.

Image

Thursday, November 14, 2019

Running dcm4che storescp tool

Dcm4che comes with a compact java implementation of the Store SCP.

You may easily download and run this tool by following:


$ wget -O dcm4che-5.19.0-bin.zip https://downloads.sourceforge.net/project/dcm4che/dcm4che3/5.19.0/dcm4che-5.19.0-bin.zip?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fdcm4che%2Ffiles%2Fdcm4che3%2F5.19.0%2Fdcm4che-5.19.0-bin.zip%2Fdownload&ts=1573680007


$ unzip dcm4che-5.19.0-bin.zip

$ cd dcm4che-5.19.0/bin

$ ./storescp --accept-unknown --directory /opt/localdrive/dcm4che-dicom-root  --filepath {00100020}/{0020000D}/{0020000E}/{00080018}.dcm -b BMIPACS:4242