Add instructions to landing page on how to get GrimoireLab running with Docker-Compose

[GeorgLink]

@sanacl created a docker-compose solution for quickly getting GrimoireLab up and running with a dashboard. The solution offers a good overview of the components that need to be running (e.g., Elasticsearch, MariaDB, and several GrimoireLab components). The solution also has a transparent way to configure the data sources and API tokens.

I think this docker-compose solution provides a good overview of the project and is fairly easy to use.

Therefore, I suggest that we replace the current single-container solution that integrates all components into on and hides the complexity but also increases the barrier to configuring it.

Ideally, we can have at the top of the landing page (README.md?) quick install instructions. A different page can contain more detailed instructions on how the docker-compose solution is setup and what the different components are and how to go in and change stuff.

I think this issue concerns changing also: https://chaoss.github.io/grimoirelab-tutorial/basics/dockerhub.html

[canasdiaz]

@GeorgLink maybe it is a good idea to agree a TOC to update the https://chaoss.github.io/grimoirelab-tutorial/. Example, I've found the section to install Grimoirelab, but it is not very visible at it is quite verbose https://chaoss.github.io/grimoirelab-tutorial/basics/install.html

I would create a section named "Installation" and if we wanna keep all the possibilities I would split them in different pages, so the docker-compose approach would be in a single page with the steps we have now (extended and improved)

[GeorgLink]

Yes, agreeing on a TOC would be good. I was looking for good examples that we can model our tutorial after.

How about a similar structure to Elasticsearch tutorial:

• Getting Started
  • Basic Concepts
  • Installation <-- this is where docker-compose goes into; end with showing that the dashboard loads (requires a data source without API token as default)
  • Setting up your projects <-- this is where we describe how to configure the setup.cfg and projects.json file
    • GitHub
    • GitLab
    • ... one sub-page for each data source
  • Managing identities <-- explaining Sortinghat and Hatstall
    • Hatstall
    • Sortinghat
  • Dashboards <-- how to create visualizations and set up dashboards
    • All data fields across all indexes
• Expert documentation
  • (focusing on installing GrimoireLab from scratch, using pip3)
  • ... this is where a lot of the current tutorial about the different components can go.

[canasdiaz]

@GeorgLink I like your proposal 👍

Given the fact we are not collection more feedback, what is the next step? Should I start working on the PR?

[GeorgLink]

I think a PR would a good place to continue the work.

@jgbarah did you have thoughts on revising the tutorial as proposed above?

[canasdiaz]

@jgbarah ?

[jgbarah]

Yes, but I need some spare cycles to process it... :-( I'll try to so asap. Sorry for being late with this.

[canasdiaz]

This ticket is blocked by #93

[GeorgLink]

In #93, we decided on a new TOC.
#104 is for implementing that new TOC. It is important to keep in mind that we want to build the new tutorial around the docker-compose approach detailed above.

[jsmanrique]

I think we are starting to have too many actions or issues around the same topic (like chaoss/grimoirelab#220).

@GeorgLink I wouldn't recommend to make everything go around the docker-compose installation option, because my perception is that it seems more simple because it's better explained and placed in a README, while the docker run grimoirelab option is lost in the middle of the tutorial with an unclear explanation about configuration files...

Having said that, my recommendation would be to have a //Grimoirelab quick start// based on 2-3 options:

• pip packages
• docker run
• docker-compose

The 3 of them must be based in or using:

• same grimoirelab version (//latest// if possible)
• same default projects.json file (just 1-2 git repositories to start)
• same setup.cfg file

Next section should be a short description of the architecture with links to each component README that would include detailed documentation, and even extra docs (using its repo /docs folder), for maintainers, contributors, and developers.

Another interesting approach would be to have 2 tutorials (but this might be more effort):

• one for GrimoireLab end users, with some or none, technical (python, docker, etc.) background
• one of GrimoireLab developers, or people with strong technical background willing to extend, or reuse GrimoireLab components (that it seems to be existing tutorial aim)

Comments?

[GeorgLink]

I have not fully developed my thoughts on whether to combine user and developer instructions in one tutorial or split them into separate tutorials. My first thought is to have different sections in a single tutorial. A second thought is to have the tutorial focus on Devs and have the GL website explain how to use the tool as a user. I can agree with having the tutorial more open and using docker-compose only for a quick-start / quick-win. Based on my experience having tried all three options, I recommend for the quick-start / quick-win page: 1. Make "docker-compose" the first option. It is the easiest to understand and separates out some of the different components (e.g., separate Elastic Search container). 2. Make pip install the second option. There are several assumptions (prerequisites) that need to be met and it requires more explanation. That's why I would put pip second. Pip is not the fast and easy win that docker-compose is. 3. Eliminate the single docker image with everything in one image. I find it more difficult to understand than the docker-compose solution. I assume it is more work to update and maintain the image where we not only have to maintain GL but also all dependencies like elasticsearch. I vote to have only one container solution and to focus on the docker-compose only.

[canasdiaz]

From my point of view the better way to attract people to our community is to offer one simple method of using the tool and provide the knowledge to go deeper as soon as the tool shown its capabilities. This method should be only one and it should be simpler than the ones we have in all the methods we've built so far. My recommendation here is to publish something based on a set of docker containers, run by a docker-compose and using as a glue a set of small scripts to make easier the setup of the demo (and just with demonstration purposes). So, I do agree with @GeorgLink in the three proposals above.

Related to the tutorials, I would avoid having two. First because it will mislead people and second because it will be hard to be maintained.

[jsmanrique]

I agree to avoid 2 separate tutorials but a general tutorial with links to components details (in their own repos README and docs folder) for those interested in deeper details, or start building their own stuff on top of them.

Regarding pip, docker, or docker-compose, if you really think docker-compose option is the right choice for a quick start, then, let's put it in the README.

IMHO, if GrimoireLab had HatStall and Bestiary fully operative, nothing would be easier than just a single docker command:

1. Requirements: Docker
2. $ docker run grimoirelab/grimoirelab
3. Go to https://localhost/grimoirelab and start adding projects, repositories...

But, this would be a longer discussion since there are people working on Cauldron (alpha) and none in Bestiary and Hatstall 😞

[valeriocos]

I agree to avoid 2 tutorials. @sanacl It depends which kind of people you want to attract. Cauldron could be useful for attracting non-technical people, docker and docker-compose are good for people with some knowledge on docker technologies (probably @jgbarah can share his point of view about the advantage of the installation through docker), pip is perfect for pythonists.

We could have a section called installation with different entry points, so we are sure to reach different types of users.

[GeorgLink]

I like the idea of having only docker-compose install method (or @jsmanrique's terraform/ansible scripts) on the first page of the tutorial. We can have a sub-page for the pip install as an alternative way of installing the components and for someone who wants to understand the different parts to grimoirelab. I agree to put the "quick-install" for docker-compose in the github.com/chaoss/grimoirelab/README.md. Would you like me to create a pull request? I would use a stripped down version of my blog post?

[valeriocos]

Ok from my side @GeorgLink .

You can reuse the content of https://github.com/chaoss/grimoirelab-tutorial/blob/agamotto/getting_started/installation.md (maybe including also the troubleshooting section). As setup.cfg and projects.json we could use the ones for CHAOSS.

WDYT?

[jsmanrique]

I am getting lost with these documentation discussions... Since we are talking in this ticket about chaoss/grimoirelab repo, I get confused. I think current discussion should be done in chaoss/grimoirelab#220

[GeorgLink]

Thank you @jsmanrique for #221.

[valeriocos]

Moving the discussion to chaoss/grimoirelab#220