redmine2github

Scripts to migrate redmine tickets to github issues. This is for a 1-time move--so it's a mix of automation and manual decisions. e.g., Get-it-done, but make the process repeatable.

Setup with virtual environment

Install pip

• python3 -m pip install --user --upgrade pip

Run venv in the project directory

• python3 -m venv env
• source env/bin/activate

Install required packages

• pip install -r requirements/base.txt

Make a config file

• Copy "/src/settings/local_sample.py to "/src/settings/local.py" and fill in variables
• Fill in your redmine and github credentials
  • For the Redmine API key, see "You can find your API key on your account page..."
  • For github, enter your credentials. (To do, use a Personal API token)

Workflow

(1) Download your open redmine issues

• Each issue is saved as a .json file, including relevant "children", "journals", "watchers", "relations"
  • The file naming convention is by ticket issue number.
    • e.g. Issue 387 becomes "00387.json"
  • Files are saved to the following directory:
    • from settings/local.py: (REDMINE_ISSUES_DIRECTORY)/(current date in 'YYYY_MMDD' format)
      • e.g. ../working_files/redmine_issues/2014-0709/(json files here) ../working_files/redmine_issues/2014-0709/03982.json ../working_files/redmine_issues/2014-0709/04050.json
• A json dict is saved to "issues_list.json" that maps the redmine issue number to the issue subject. For example: * e.g. ../working_files/redmine_issues/2014-0709/issues_list.json

    {
        "03982": "Create Genomics Metadata Block", 
        "04050": "Additional Astronomy FITS File Metadata Support for Units", 
        "04051": "Metadata: Astronomy - Changes to Ingest and Display of Resolution Elements", 
        "04072": "Edit Dataverse: Checking/unchecking use facets from Host Dataverse undoes any changes in the rest of the form."
    }
    

Example of downloading redmine issues

• cd into the src/redmine_ticket directory
• update the bottom of the "redmine_issue_downloader.py" file
• Currently looks something like this:

if __name__=='__main__':
    from settings.base import REDMINE_SERVER, REDMINE_API_KEY, REDMINE_ISSUES_DIRECTORY
    #rn = RedmineIssueDownloader(REDMINE_SERVER, REDMINE_API_KEY, 'dvn', REDMINE_ISSUES_DIRECTORY)
    rn = RedmineIssueDownloader(REDMINE_SERVER, REDMINE_API_KEY, 1, REDMINE_ISSUES_DIRECTORY)
    rn.download_tickets()

• run it:

../redmine2github/src/redmine_ticket> python redmine_issue_downloader.py

(2) Migrate your issues to a github repository

---

Note 1: Once added, a github issue cannot be deleted. Therefore, to test your migration, create a new "scratch" github repository. Once you're satisfied that the migration works, delete the scratch repository.

---

---

Note 2: The current GitHub API limit is 5,000/day. Adding each issue may have 1-n API calls. Plan appropriately.

• 1 API Call: Create issue with labels, milestones, assignee
  • This process creates a json file mapping { Redmine issue number : GitHub issue number}
• 0-n API Calls for comments: A single API call is used to transfer each comment
• 2 API Calls for related issues (optional): After all issues are moved
  • Call 1: Read each GitHub issue
  • At the bottom of the description, use the Redmine->GitHub issue number mapping to add related issue numbers and child issue numbers
  • Call 2: Update the GitHub description

---

Quick script

• cd into the src/github_issues directory
• update the bottom of the "migration_manage.py" file
• Currently looks something like this:

if __name__=='__main__':
    # e.g. json_input_directory, where you downloaded the redmine JSON
    #      json_input_directory="some_dir/working_files/redmine_issues/2014-0709/"
    #
    json_input_directory = os.path.join(REDMINE_ISSUES_DIRECTORY, '2014-0702')

    kwargs = dict(include_comments=True\
                , include_assignee=False\
                , redmine_issue_start_number=4123\
                , redmine_issue_end_number=4134\
                , user_mapping_filename=USER_MAP_FILE       # optional
                , label_mapping_filename=LABEL_MAP_FILE     # optional
                , milestone_mapping_filename=MILESTONE_MAP_FILE # optional
            )
    mm = MigrationManager(json_input_directory, **kwargs)
    mm.migrate_issues()

• run it:

../redmine2github/src/github_issues>python migration_manager.py

Label Map Notes

The label map is optional. It allows you to assign label names and colors by creating a label map file.

• See Sample Label Map, sample_label_map.csv

---

• The redmine_type column is for user convenience only, it is ignored by the program. So watch out for name collisions

---

• Pertains to Redmine name values in fields status, tracker, priority, or custom_fields
• If no map is specified in the MigrationManager kwargs:
  • The status, tracker, priority, or custom_fields names in Redmine issues are made into GitHub labels. See "def get_label_names" in the label_helper.py file
    • A Redmine status of "New" would turn into label "Status: New"
    • A Redmine tracker of "Feature" would turn into label "Tracker: Feature"
    • A Redmine priority of "Urgent" would turn into label "Priority: Urgent"
    • A Redmine custom field of "UX/UI" turns into label "Component: UX/UI"
  • If no map is specified, then newly created labels will not have a color

Map Notes - How is the map used?

• The map is specfied in the settings/local.py file under LABEL_MAP_FILE
• If a status, tracker, priority, or custom_field name in a Redmine ticket is NOT found in the map, that name value will NOT be moved to GitHub
• The map file is "dumb." If you would like to map more than one status name to a single status label, simply repeat it.
  • In the example below, the "redmine_name"s "In Design" and "In Dev" are both mapped to the label named "Status 3: In Design/Dev"
  • For repeated github_label_names, make sure they're the same:)

redmine_type, redmine_name, github_label_name, github_label_color
status, In Design, Status 3: In Design/Dev,996600
status, In Dev, Status 3: In Design/Dev,996600

• When the map is read, the values are trimmed. e.g. ", In Design ," would become "In Design" with leading/trailing spaces removed