Thursday, December 5, 2019

Running Kheops on a Mac OS host

This post discusses how to run Kheops from the on Mac OS, with Docker.

$ git clone git@github.com:OsiriX-Foundation/kheopsDocker.git

$ cd kheopsDocker/

$ sh script_run_docker.sh

Install newman

$ sudo npm install -g newman

/usr/local/bin/newman -> /usr/local/lib/node_modules/newman/bin/newman.js
+ newman@4.5.6
added 160 packages from 196 contributors in 5.481s


   ╭────────────────────────────────────────────────────────────────╮
   │                                                                │
   │       New minor version of npm available! 6.9.0 → 6.13.2       │
   │   Changelog: https://github.com/npm/cli/releases/tag/v6.13.2   │
   │               Run npm install -g npm to update!                │
   │                                                                │
   ╰────────────────────────────────────────────────────────────────╯



Replace line 10 in script_run_docker.sh

with

newman run postman/kheops-Get_tokens.json -e postman/kheops_localhost.postman_environment.json -g postman/Workspace.postman_globals.json --export-globals postman/Workspace.postman_globals.json

Save the file.

If you are using Docker for Mac, you may encounter the error listed in https://github.com/docker/for-mac/issues/2396

Creating pacsldap ... error
Creating db       ...

ERROR: for pacsldap  Cannot start service ldap: b'Mounts denied: \r\nThe paths /etc/localtime and /usr/share/zoneinfo/Europe/Zurich\r\nare not shared from OS X and are not known to Docker.\r\nYou can configure shared paths from Docker -> Preferences... -> File Sharing.\r\Creating db       ... done

ERROR: for ldap  Cannot start service ldap: b'Mounts denied: \r\nThe paths /etc/localtime and /usr/share/zoneinfo/Europe/Zurich\r\nare not shared from OS X and are not known to Docker.\r\nYou can configure shared paths from Docker -> Preferences... -> File Sharing.\r\nSee https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.\r\n.'
ERROR: Encountered errors while bringing up the project.

As a work-around, remove all the occurrences of the below line from kheops/docker-compose.yml.
      - /etc/localtime:/etc/localtime:ro

$ sudo mkdir /dcm4chee-arc

$ sudo chmod 777 /dcm4chee-arc

Share /usr/share/zoneinfo/Europe/ and /dcm4chee-arc from the Preferences of Docker Desktop in your Mac host machine.

Save the file and then re-run

$ sh script_run_docker.sh

$ sh script_run_docker.sh
Stopping db       ... done
Stopping pacsldap ... done
Removing pacsarc  ... done
Removing db       ... done
Removing pacsldap ... done
Removing network kheops_default
Removing volume kheops_dcm4chee-ldap-ldap
Removing volume kheops_dcm4chee-ldap-slapdd
Removing volume kheops_dcm4chee-db-data
Removing volume kheops_dcm4chee-arc-wildfly
Removing volume kheops_kheops-db-data
Creating network "kheops_default" with the default driver
Creating volume "kheops_dcm4chee-ldap-ldap" with default driver
Creating volume "kheops_dcm4chee-ldap-slapdd" with default driver
Creating volume "kheops_dcm4chee-db-data" with default driver
Creating volume "kheops_dcm4chee-arc-wildfly" with default driver
Creating volume "kheops_kheops-db-data" with default driver
Creating db       ... done
Creating pacsldap ... done
Creating pacsarc  ... done
Creating kheopszipper           ... done
Creating kheopsauthorization    ... done
Creating pacsauthorizationproxy ... done
Creating kheopsui               ... done
Creating kheopsdicomwebproxy    ... done
Creating kheopsreverseproxy     ... done
CONTAINER ID        IMAGE                                              COMMAND                  CREATED             STATUS              PORTS                                                                                                                                              NAMES
d5caa14fac54        osirixfoundation/kheops-reverse-proxy:ci-test      "./etc/nginx/conf.d/…"   47 seconds ago      Up 45 seconds       0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:8042->8042/tcp, 0.0.0.0:8048->8048/tcp                                                           kheopsreverseproxy
67552e9c9469        osirixfoundation/kheops-dicomweb-proxy:master      "./replaceSecretsAnd…"   48 seconds ago      Up 46 seconds       0.0.0.0:8049->8080/tcp                                                                                                                             kheopsdicomwebproxy
92710a77dd25        osirixfoundation/kheops-ui:master                  "./docker-entrypoint…"   48 seconds ago      Up 46 seconds       80/tcp, 3000/tcp                                                                                                                                   kheopsui
c9fa33eb693d        osirixfoundation/kheops-authorization:dev          "./replaceSecretsAnd…"   50 seconds ago      Up 47 seconds       8080/tcp                                                                                                                                           kheopsauthorization
0f267500942f        osirixfoundation/kheops-zipper:master              "./replaceSecretsAnd…"   50 seconds ago      Up 48 seconds       8080/tcp                                                                                                                                           kheopszipper
c95c6e4380c6        osirixfoundation/pacs-authorization-proxy:master   "/bin/sh -c '. $NGIN…"   50 seconds ago      Up 48 seconds       0.0.0.0:82->80/tcp                                                                                                                                 pacsauthorizationproxy
89c52e69534a        osirixfoundation/kheops-dcm4chee-arc-psql:latest   "/script_entrypoint.…"   51 seconds ago      Up 49 seconds       0.0.0.0:2575->2575/tcp, 0.0.0.0:8080->8080/tcp, 0.0.0.0:8443->8443/tcp, 0.0.0.0:8787->8787/tcp, 0.0.0.0:9990->9990/tcp, 0.0.0.0:11112->11112/tcp   pacsarc
78bf11251bf7        dcm4che/slapd-dcm4chee:2.4.44-15.0                 "/docker-entrypoint.…"   52 seconds ago      Up 50 seconds       389/tcp                                                                                                                                            pacsldap
f4628a1a1c44        osirixfoundation/kheops-database:demo              "verify-secret-env-v…"   52 seconds ago      Up 50 seconds       5432/tcp                                                                                                                                           db
newman

kheops-Get_tokens

→ well-known
  GET https://keycloak.kheops.online/auth/realms/travis/.well-known/openid-configuration [200 OK, 2.62KB, 657ms]

→ get token user 1 (titi)
  POST https://keycloak.kheops.online/auth/realms/travis/protocol/openid-connect/token [200 OK, 2.01KB, 325ms]

→ get token user 2 (toto)
  POST https://keycloak.kheops.online/auth/realms/travis/protocol/openid-connect/token [200 OK, 2.01KB, 314ms]

→ get token user 3 (tata)
  POST https://keycloak.kheops.online/auth/realms/travis/protocol/openid-connect/token [200 OK, 2.01KB, 332ms]

┌─────────────────────────┬─────────────────────┬────────────────────┐
│                         │            executed │             failed │
├─────────────────────────┼─────────────────────┼────────────────────┤
│              iterations │                   1 │                  0 │
├─────────────────────────┼─────────────────────┼────────────────────┤
│                requests │                   4 │                  0 │
├─────────────────────────┼─────────────────────┼────────────────────┤
│            test-scripts │                   4 │                  0 │
├─────────────────────────┼─────────────────────┼────────────────────┤
│      prerequest-scripts │                   0 │                  0 │
├─────────────────────────┼─────────────────────┼────────────────────┤
│              assertions │                   0 │                  0 │
├─────────────────────────┴─────────────────────┴────────────────────┤
│ total run duration: 1790ms                                         │
├────────────────────────────────────────────────────────────────────┤
│ total data received: 7.53KB (approx)                               │
├────────────────────────────────────────────────────────────────────┤
│ average response time: 407ms [min: 314ms, max: 657ms, s.d.: 144ms] │
└────────────────────────────────────────────────────────────────────┘

Then go to http://localhost:8042 from your favorite browser.

Register yourself when it redirects to a page such as https://keycloak.kheops.online/auth/realms/travis/protocol/openid-connect/auth?client_id=loginConnect&redirect_uri=.........

It will take you back to http://localhost:8042/inbox after the registration. Now Kheops is ready to use!
Kheops Web UI

I can even tweet from the Kheops Web UI with a link to the created album!
"My KHEOPS shared album. http://localhost/view/ttuArLPrFv834JQ88KaQGY #KHEOPS"
Apparently, the link is broken. It should be: http://localhost:8042/view/ttuArLPrFv834JQ88KaQGY
But I see the parallels with the concept of Óbidos (MEDIator) replicasets in the given UUID ttuArLPrFv834JQ88KaQGY.
One can also create a sharing URL with upload/edit/delete capabilities (again, the port is not correctly used. But that is a front-end bug they can fix. For our purposes, we just need to add the port in the shared URL).

I can also view the images that I uploaded from OHIF viewer directly, by clicking that link. It uses their cloud hosted instance of the OHIF viewer to view the image stored locally, as can be seen from the URL.

Monday, December 2, 2019

Those who ruin the flights for others

Batu Caves Murugan
The last week I had a long return flight to Kuala Lumpur from Atlanta, via Doha. Usually, I watch a lot of movies during the flights. But this time, I decided to watch just 2 movies during the entire journey (to and from).

In any economy class, there are 3 (or 4, if you sit in a middle seat) people who can ruin your flight. First, the person who sits in the seat behind you unless you sit in the last row. Second, the person who sits in front of you, unless you sit in the exit row or the first row in the block. Finally, the person(s) who sit(s) next to you.

The person who sits behind you has the potential to ruin your journey entirely. They can kick your seat non-stop. Some people do not realize how weak the economy class seat is. If you put your feet on the seat tray, or if you just constantly jerk it, the movements entirely affect the seat in front of you. It gives headaches. Please don't do that. Infants have the habit of kicking. If possible, make sure their feet are not directed towards the seat in front of them.

Second, the person who sits in front of you. They may recline their seat any time, giving your knee a sudden jerk. Some idiots choose to do that while the meals are served, making you spill your food or drink on you. I usually avoid reclining my seat (except maybe by a negligible degree). One exception: if you are in the seat behind me and continue to kick or jerk my seat, I WILL recline my seat to the fullest.

Finally, the person who sits next to you. They may occupy your seat area, lean on you and sleep, or cross their leg and touch you with their shoe or smelly socks. There was a time I was sandwiched in a middle seat with people in either side (window and aisle seats) occupying both handrests. People, please give that armrest to the middleseater! On the other hand, once in a flight, as always, I let the middle-seat person use the entire armrest. But after a few minutes, I noticed I was being pushed to a corner. Then I found that that person has folded the armrest, so that they can invade my space! Of course, I agree that they are a big person. However, it is NOT okay. I instantly put the armrest down back in its place, effectively cutting the huge person's belly which was invading my space by 25%. The person realized their mistake and retracted.

I wish everyone tries to be mindful of the surroundings and the neighbours. This will make our economy class flight experience much better.

Sunday, November 17, 2019

Revising Portuguese with Duolingo

Since I successfully defended my PhD thesis this year and left Portugal, I started to miss Portugal and hearing Portuguese. I also do not want to forget Portuguese, the language of the country that I love most. Therefore, I started practicing Portuguese with Duolingo 10 days ago. So far, good progress. Duolingo is funny with its interesting lessons and stories.

Some interesting events from Duolingo.

1. Luis is single-handedly responsible for the pineapple shortage in the world.
Image 


2.  and this woman who can eat eighteen! sandwiches.
Image 
3.  Proud to live in a world where even the cats read books.
Image 
4. and it gets creepier..
Image


5. Tell me about these unfair expectations..
Image 
6. A vegan tiger.

7. The mythical armadillo strikes again.. :D
Well, duolingo has a thing with this tatu.

Image

Thursday, November 14, 2019

Running dcm4che storescp tool

Dcm4che comes with a compact java implementation of the Store SCP.

You may easily download and run this tool by following:


$ wget -O dcm4che-5.19.0-bin.zip https://downloads.sourceforge.net/project/dcm4che/dcm4che3/5.19.0/dcm4che-5.19.0-bin.zip?r=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fdcm4che%2Ffiles%2Fdcm4che3%2F5.19.0%2Fdcm4che-5.19.0-bin.zip%2Fdownload&ts=1573680007


$ unzip dcm4che-5.19.0-bin.zip

$ cd dcm4che-5.19.0/bin

$ ./storescp --accept-unknown --directory /opt/localdrive/dcm4che-dicom-root  --filepath {00100020}/{0020000D}/{0020000E}/{00080018}.dcm -b BMIPACS:4242

Saturday, November 2, 2019

Universities as mentoring organizations

A fraction of the session participants.
(Photo by akram9)
We had several exciting unconference sessions and talks at the Google Summer of Code Mentor Summit 2019. I proposed and coordinated the unconference session titled "Universities as mentoring organizations," the Sunday 20th of October 9:00 - 10:00 a.m., at the room "Studio-2" of Mariott Munich. We had around 25 active participants from multiple organizations, representing several universities from countries including the USA, Russia, Germany, and Brazil. We also had mentors from umbrella organizations such as CBioPortal, which are based mainly on universities and research labs. Later on the same day, I coordinated a session on "The great proposals" at "Studio-3" from 2:30 - 3:30 p.m.
 
The session notes are recorded in the GSoC notes. Find all the sessions with their notes linked here. In this post, I elaborate on the session "Universities as mentoring organizations" again for a wider audience. These notes are from the thoughts of the mentors from the communities involved in the discussion, and they will reflect the communities involved. Discussions are grouped under topics, rather than by the person who discussed it in detail.


Administrative Challenges from the Universities
The discussions started interactively with self-introductions from the participating mentors and their mentoring organizations. Tiago and Frederico represented their mentoring organization, which is a university in Brazil. They noted the challenges in convincing the university administration to join the GSoC and then accepting payments from Google. Their organization was the first Brazilian university to become a GSoC mentoring organization. The university administration was expecting documents from Google in Brazilian Portuguese to consider them official. The universities expected the supporting documents could come from one of Google's Brazil offices rather than from Mountain View to make the language requirement met. This expectation created some translation requirements, and these cause some additional burden on the GSoC organization administrators. We hope that such issues would be sorted out from the universities, potentially with some help from Google, so that there would be more participation from International (i.e., non-English-speaking) universities as mentoring organizations.

GSoC Students Help Ph.D. Research
There were several observations on how GSoC students help with the implementation of research ideas. Several Ph.D. implementation works remain closed due to implementations with significant technical debt. Support from GSoC helps make the code more readable and reusable, and thus support open science and open-source contributions from the research universities. Hence GSoC students help with the host universities' research open-source. Stephanie from UC Santa Cruz stressed how GSoC helped their Ph.D. and postdoc researchers. She mentioned how the undergrad GSoC students worked with their researchers, and this was a mutually benefitting task. No one felt mentoring as a burden. Instead, they saw it as a way to build their communities. She highlighted that promoting GSoC across the departments would be easy by stressing how the GSoC students help with implementation over the summer, paid by Google. There were observations on developing projects that are helpful to Ph.D. research work. For instance, there are projects that need to get done, but the mentor (Ph.D. student) doesn’t want to do it themselves. GSoC students can be more motivated to do such implementation work, even with little scientific or research impact.

Motivating students into GSoC
Philipp and Karlheinz represented the University of Munich. They highlighted that their Ph.D. students work as GSoC mentors. However, they also stressed that it is hard to motivate students from their university to join GSoC as students due to summer holidays overlapping with the GSoC timeframe. Students do go off on a well-deserved vacation rather than taking a summer internship. Furthermore, there were observations that there are several university open source projects that are not connected to or well-received by general open-source software communities. GSoC is indeed bridging this gap, making the code quality better. I (Pradeeban from EmoryBMI - Emory University, Department of Biomedical Informatics) mentioned how we motivate our students to join GSoC while discouraging them from joining our university/department as their mentoring organization. This is mainly because we want them to build new collaborations. Collaborations inside the university or department do not require a GSoC.

GSoC in professional life
The impact of GSoC is long-lasting. Some of us found our postdoc advisors and employers through GSoC. So GSoC works as a recruiting platform for mentors as well. Furthermore, it fosters international collaborations between universities. We had research papers as outcomes of the GSoC. Nikita from a Russian university stressed the importance of GSoC and similar programs in the students' careers. Sebastian Diaz from Harvard highlighted the misalignment of university projects and open source.  While challenging, programs such as GSoC help fill this gap. Tobias highlights that their target students mostly include Ph.D. students, as they are more suited to work on the proposed projects, involving significant research component. GSoC also helps them get more funding for full-time developers, as it is considered a full-time job for the students.

Best Students Join the Universities
Aadi, a high school student from India, observed how the best students like to work with the universities that function as GSoC mentoring organizations, as this helps them with their future graduate studies and research. The observation is that the mentors from such organizations are professors and experienced researchers from the universities. This is also a mutually beneficial relationship - universities as mentoring organizations get to have the best students. Indeed, a win-win. I joined OMII-UK as a student in 2010 and Emory BMI in 2014 and 2015 (2016 onward as a mentor and then joined 2018 as an employee, and 2019 a postdoc). My interest for OMII-UK was driven by the fact that the EPCC research team from Edinburgh University was part of OMII-UK, where I worked with them on their OGSA-DAI platform for GSoC 2010.

Why not enough open-source from universities?
Arav Singhal, a student from Rice University, stressed he prefers more open-source development in his university. We agreed that a large number of research teams keep their source closed until their paper is published. Even after that, when the code is made public, most of the time, the code is not reusable, as it is usually developed as a prototype with little attention to code readability and engineering best practices. GSoC helps fix this by building up coding skill sets in the students.

Small open source presence in a country
Deniz is from Turkey. He was a GCI student in ScoReLab (an open-source community originated from the University of Colombo, Sri Lanka). He highlighted how Turkey has a small open-source presence. The lack of local open-source communities makes motivating students to join GSoC harder. The challenge starts with introducing open-source and then GSoC, among the potential students. It is essential to create larger local open source communities to build a diverse expanding FOSS contribution.

Universities/Entities collaborating as a single mentoring organization
Werner highlighted the collaboration between multiple universities as a mentoring organization. He observes that sometimes, organizations do not make it explicit enough that the organization is a research department or a collaboration of such entities. Ino from cBioPortal highlighted how hospitals and research institutes collaborate under cBioPortal for the GSoC. He also noted the positive outcomes, including peer-reviewed publications. Mentioning GSoC in CV - both as a student and a mentor - is rewarding. There were suggestions on including open source software development in university curriculums.


Starting a new GSoC organization
Akram, a researcher from the University of Tennessee (Dept. of Bioinformatics), helped his department apply and get selected as a mentoring organization as a new (i.e., first-time) GSoC organization when he moved to the current university as a postdoc from another university. He observes that the GSoC has indeed resulted in peer-reviewed journal publications. He stressed the importance of open source for university. There seems to be a common observation that receiving funding from Google (or any similar company) becomes harder due to the university regulations. We discussed how to start as a mentoring organization. We need to have precise project ideas. Of course, having established open source projects would be a big plus. However, there are also concerns about having mentors and retaining them. How to grow our mentoring community beyond the walls of the department/university? GSoC students-turning-mentors can be a solution. We also need to be clear on the bioinformatics side on what can be open source, as we deal with sensitive health data.

Recruiting Mentors
One challenge to address is how to leverage more departments/mentors from the university. Some of us view it as building the community. This requires a significant effort from the organization administrator, to convince the fellow faculty, postdocs, researchers, and staff to be GSoC mentors. Sometimes, it is faster just to do the development ourselves, rather than mentoring a student to do it. However, such mentoring can be a rewarding experience for early career researchers. We all agreed that GSoC was a productive use of our time and not overhead. We also need a "mentor pool." Projects should have back-up mentors and an active co-mentors. This helps with avoiding mentor burn-out, ensuring successful completion of GSoC.

GSoC as a funding mechanism
Some view GSoC projects as a way to fund implementation activities that other funding entities won’t financially support. For instance, development is often seen as not novel. Therefore, complete implementation and maintenance efforts do not receive sufficient funding from the funding agents. However, such maintenance and incremental developments are crucial for the usability of the project. GSoC helps improve the usability and maintenance of the code.


Did we miss anything? Also, did I fail to include any crucial aspects discussed in the session? Please share your thoughts on this topic as comments.