Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrating mySQL database into strapi #104

Open
hanEck opened this issue Aug 16, 2024 · 3 comments
Open

Migrating mySQL database into strapi #104

hanEck opened this issue Aug 16, 2024 · 3 comments
Labels
enhancement New feature or request

Comments

@hanEck
Copy link
Collaborator

hanEck commented Aug 16, 2024

Currently the VIM websites has a mysql database, which has to be migrated to strapi. The columns of interest for a first migration are the users and the scripts. Following thoughts/discussions have been made:
=> Users could be migrated on the fly => Additional requirement is to use a new hashing algorithm for the password
=> There script tables of interest for a first migration are: vs_scripts, vs_script_source, vs_script_ratings, vs_script_downloads
=> Topic is also strong related with seeding

Issues/concerns:
=> When migrating scripts, there is a direct connection to the users
- This is critical if the user migration is done on the fly => All scripts should be available from the beginning even though only a limited amount of users is migrated
- Possible solution: Adding placeholder users for not migrated users and only actually migrate and connect them to scripts in case of on the fly migration (e.g. first login)

@hanEck hanEck added the enhancement New feature or request label Aug 16, 2024
@Shane-XB-Qian
Copy link

Shane-XB-Qian commented Aug 16, 2024 via email

@georgalis
Copy link

Apologies for the boat load of information here, in this LONG comment. I started following this repo because I wanted to follow the progress, I'm familiar with many dimensions of the challenges involved. My first impression was, "it will be difficult to help here" because the ./README.md reads more like napkin proposal (an important step BTW!) than a plan, and there is no reasoning is included, eg due to licensing and open source momentum, one would expect postgres to be the db of choice?

My point is not which db to use, but this issue got my attention. It looks like strapi does support mysql https://docs.strapi.io/dev-docs/configurations/database but probably not the way it is a requirement.

I appreciate there are a lot of opinions, and noise, with regard to undertaking a large project like this and I don't mean to distract! Although I want to contribute incrementally, I have no idea of the actual design, status, or workflow of the website refresh, and I anticipate many more experienced than myself are in the same situation. So my comment is about developing the framework for the design development and presentation to better crowd source from casual observers like myself.

This is more of an inventory of ingredients that might contribute to an improved project development workflow, with a key goal of better solution crowd sourcing. As it may become be obvious, this solution was LLM developed, but please don't discredit it for being synthetic, I spent over an hour drafting the prompt (substantially longer than this introduction) to address the challenges I'm observing here.

As a next step, I recommend going through the framework and annotating the line items with their applicability. Some items can easily be excluded with "too much control engineering" but I suggest going a step further to identify how the consequence of exclusion would play out, to better anticipate and pre-establish mitigations. Simply using the github interface, is not the solution, as elucidated in Is there a better project management method than maintaining the todo.txt manually? #14974 Many different project development frameworks could be highly effective. The key is establishing/publishing one, proactively selecting the steps relevant to the project, and in this case enabling efficient input from transient contributors. In my view that means defining a transparent process, recording the workflow in the repo, alongside the design and a record of solution evaluations, and lessons learned.

Yes there is some overhead there, the benefit might not become apparent for six months, or when onboarding a new contributor, or another one, or just crowd sourcing a solution detail, all of these become more efficent when the larger context is presented in a more consumable manner.

Thanks,
-George


Subject: Introducing a New Collaborative Framework for Our Website Transition

Dear Contributors,

We've recently encountered some challenges in our website CMS transition project, particularly regarding conflicting requirements and difficulty in coordinating our distributed team's efforts. In response, we're proposing a new collaborative framework to streamline our development process and enhance our ability to leverage the diverse expertise within our community.

This framework aims to:

  1. Clearly classify and document our requirements
  2. Establish a structured workflow for evaluating solutions
  3. Improve transparency in our design and decision-making processes
  4. Facilitate easier onboarding and contribution for new team members
  5. Enable more effective asynchronous collaboration

We believe this approach will help us overcome our current roadblocks and create a more robust, community-driven development process. In the following sections, you'll find a detailed breakdown of this framework, including its components, implementation plan, and expected benefits.

We welcome your feedback and suggestions as we work together to refine and implement this new strategy.


I've created a detailed framework to address the challenges and requirements you've outlined for transitioning the open-source project's website CMS. This framework is designed to facilitate transparent, collaborative development while accommodating contributors with varying levels of involvement.

Key aspects of the proposed solution include:

  1. A structured approach to requirements management, including clear classification and documentation.
  2. A defined workflow for evaluating potential solutions against requirements.
  3. Comprehensive design documentation to capture current status, rationale, and future plans.
  4. Progress tracking tools to visualize the project's status and upcoming tasks.
  5. A collaborative platform using Git-based systems for version control and documentation.
  6. Multiple communication channels to support both synchronous and asynchronous collaboration.
  7. A system for creating and evaluating Proof of Concept (POC) demos.

This framework addresses the current challenges by providing:

  1. Clear organization and classification of requirements.
  2. A structured process for vetting solutions against requirements.
  3. Transparent documentation of design decisions and project status.
  4. Tools for new contributors to quickly understand the project's current state and challenges.
  5. Methods for integrating contributions from part-time or occasional contributors.

The proposed solution empowers crowd-sourcing of innovative solutions by:

  1. Lowering the barrier to entry for new contributors through clear documentation.
  2. Enabling focused contributions by clearly defining requirements and design gaps.
  3. Allowing for rapid testing of ideas through POC demos without requiring full implementation.
  4. Providing visibility into how individual contributions fit into the larger project.
  5. Recognizing community contributions to encourage ongoing participation.

Justification for iterating working POC demos:

  1. Allows for quick evaluation of potential CMS solutions without full content migration.
  2. Reduces the risk of committing to a solution that may not meet all requirements.
  3. Enables community feedback on proposed solutions before significant resources are invested.
  4. Facilitates comparison between different options in a practical context.
  5. Helps identify potential integration issues or limitations early in the process.

By implementing this framework, the project can effectively manage its transition to a new CMS while leveraging the diverse expertise of its community. The structure provides clarity and direction, while the flexibility allows for innovation and adaptation as the project evolves.


Framework for Open Source Project Website Transition

Current Challenges

  1. Conflicting requirements between new tools and project needs
  2. Lack of clear requirements classification (must-have, nice-to-have, wish list)
  3. Absence of workflows to vet potential solutions against requirements
  4. Difficulty for new contributors to understand current status and make meaningful contributions
  5. Lack of transparency in design and decision processes

Proposed Solution: Collaborative Requirements and Design Management System

1. Requirements Management

a. Requirements Classification

  • Must-have
  • Nice-to-have
  • Wish list

b. Requirements Documentation

  • Unique ID for each requirement
  • Description
  • Rationale
  • Priority
  • Status (Implemented, In Progress, Not Started)
  • Dependencies

2. Solution Evaluation Workflow

  1. Solution Proposal
  2. Initial Screening
  3. Detailed Analysis
  4. Community Feedback
  5. Decision Making
  6. Implementation Planning

3. Design Documentation

  • Current Design Overview
  • Design Rationale
  • Known Limitations
  • Future Roadmap

4. Progress Tracking

  • Kanban Board for visual representation of tasks
  • Milestone tracking
  • Regular status updates

5. Collaborative Platform

  • Use of Git-based system (e.g., GitHub, GitLab) for version control and collaboration
  • Wiki for comprehensive documentation
  • Issue tracker for managing tasks and bugs

6. Communication Channels

  • Mailing list for important announcements
  • Chat platform (e.g., Slack, Discord) for real-time discussions
  • Regular video conferences for synchronous communication

7. Proof of Concept (POC) Demos

  • Implement a system for creating and sharing POC demos
  • Document the process for creating and evaluating POCs
  • Provide guidelines for feedback on POCs

Implementation Plan

  1. Set up the collaborative platform (Git repository, Wiki, Issue tracker)
  2. Create initial documentation structure
  3. Migrate existing requirements and design documents
  4. Establish communication channels
  5. Develop and document the solution evaluation workflow
  6. Create templates for requirements, design documents, and POC demos
  7. Train core team members on the new system
  8. Announce the new system to the community and provide onboarding resources

Benefits of the Proposed Solution

  1. Transparency: All information is accessible and version-controlled
  2. Asynchronous Collaboration: Contributors can work at their own pace
  3. Structured Decision-Making: Clear processes for evaluating and implementing changes
  4. Knowledge Retention: Comprehensive documentation of decisions and rationale
  5. Scalability: System can accommodate growing number of contributors
  6. Flexibility: Easily adaptable to changing project needs

Empowering Crowd-Sourced Innovation

  1. Lower Barrier to Entry: Clear documentation helps new contributors quickly understand the project
  2. Focused Contributions: Well-defined requirements and design gaps guide efforts
  3. Iterative Improvements: POC demos allow for rapid testing of ideas without full implementation
  4. Visibility of Impact: Progress tracking shows how individual contributions fit into the bigger picture
  5. Community Recognition: Acknowledgment of contributions encourages ongoing participation

By implementing this framework, the project can harness the collective expertise of its community while maintaining a structured approach to development. This balance of structure and flexibility will enable the project to evolve efficiently and innovatively, even with distributed, part-time contributors.

@FabianUntermoser
Copy link
Collaborator

Thanks @georgalis for your input. I can't add to the project plan since im just a contributor, but I think this topic should be moved to the discussions. Feel free to copy+paste it there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants