22 Oct 15:32

152a500

Rapid Prototyping of Agents with Hotswappable Components Pre-release

Pre-release

Roy

Roy is a lightweight alternative to autogen for developing advanced multi-agent systems using language models. It aims to simplify and democratize the development of emergent collective intelligence.

Features

Model Agnostic: Use any LLM, no external APIs required. Defaults to a 4-bit quantized wizard-coder-python model for efficiency.
Modular and Composable: Roy decomposes agent interactions into reusable building blocks - templating, retrieving, generating, executing.
Transparent and Customizable: Every method has a clear purpose. Easily swap out components or add new capabilities.

Quickstart

git clone https://github.com/JosefAlbers/Roy
cd Roy
pip install -r requirements.txt
pip install -U transformers optimum accelerate auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/

from roy import Roy, Roys
roy = Roy()
s = '"What date is today? Which big tech stock has the largest year-to-date gain this year? How much is the gain?'
roy.generate(roy.format(s))

Rapid Benchmarking

Roy provides a simple way to evaluate and iterate on your model architecture.. This allows you to:

Easily swap out components, such as language models, prompt formats, agent architectures, etc
Benchmark on different tasks like arithmetic, python coding, etc (default is OpenAI's HumanEval)
Identify agent's areas of strengths and weaknesses

from Roy.util import piecewise_human_eval

# Comparing different language models
piecewise_human_eval(0, lm_id='TheBloke/WizardCoder-Python-7B-V1.0-GPTQ') 
# -> {'pass@1': 0.6341463414634146}
piecewise_human_eval(0, lm_id='TheBloke/tora-code-7B-v1.0-GPTQ') 
# -> {'pass@1': 0.5609756097560976}
piecewise_human_eval(0, lm_id='TheBloke/Arithmo-Mistral-7B-GPTQ')
# -> {'pass@1': 0.5121951219512195}

# Testing a custom agent architecture
piecewise_human_eval(0, fx=<your_custom_Roy_agent>)

Takes around 30 minutes each on a free Google Colab runtime.

Constrained Beam Search

Use templates to structure conversations (control output length, format, etc)

roy.generate(s, ('\n```python', '\n```'))                      # Generate a python code block
roy.generate(s, (('\n```python', '\n```javascript'), '\n```')) # Generate python or javascript codes
roy.generate(s, ('\n```python', 100, '\n```'))                 # Generate a code block of size less than 100 tokens

Retrieval Augmented Generation

Enhance generation with relevant knowledge.

s = 'Create a text to image generator.'
r = roy.retrieve(s, n_topk=3, src='huggingface')
[roy.generate(s) for s in r]

Auto-Feedback

Agents recursively improve via critiquing each other.

s = "Create a secure and unique secret code word with a Python script that involves multiple steps to ensure the highest level of confidentiality and protection.\n"
for i in range(2):
    c = roy.generate(s, prohibitions=['input'])
    s += roy.execute(c)

Auto-Grinding

Agents collaborate in tight loops to iteratively refine outputs to specification.

user_request = "Compare the year-to-date gain for META and TESLA."
ai_response = roy.generate(user_request, ('\n```python', ' yfinance', '\n```'))
for i in range(2):
    shell_execution = roy.execute(ai_response)
    if 'ModuleNotFoundError' in shell_execution:
        roy.execute(roy.generate(roy.format(f'Write a shell command to address the error encountered while running this Python code:\n\n{shell_execution}')))
    elif 'Error' in shell_execution:
        ai_response = roy.generate(roy.format(f'Modify the code to address the error encountered:\n\n{shell_execution}'))
    else:
        break

Multi-Agent

Flexible primitives to build ecosystems of agents.

roys = Roys()

# AutoFeedback
roys.create(agents = {'Coder': 'i = execute(generate(i))'})
roys.start(requests = {'i': 'Create a mobile application that can track the health of elderly people living alone in rural areas.'})

# Retrieval Augmented Generation
roys.create(
    agents = {
        'Retriever': 'r = retrieve(i)',
        'Generator': 'o = generate(r)',
        })
roys.start(requests = {'i': 'Create a Deutsch to English translator.'})

# Providing a custom tool to one of the agents using lambda
roys.create(
    agents = {
        'Coder': 'c = generate(i)',
        'Proxy': 'c = custom(execute(c))',
        },
    tools = {'custom': lambda x:f'Modify the code to address the error encountered:\n\n{x}' if 'Error' in x else None})
roys.start(requests = {'i': 'Compare the year-to-date gain for META and TESLA.'})

# Another way to create a custom tool for agents
def custom_switch(self, c):
    py_str = 'Modify the code to address the error encountered:\n\n'
    sh_str = 'Write a shell command to address the error encountered while running this Python code:\n\n'
    # x = roys.execute(c)
    x = self.execute(c)
    if 'ModuleNotFoundError' in x:
        # roys.execute(roys.generate(sh_str+x))
        self.execute(self.generate(sh_str+x))
    elif 'Error' in x:
        # roys.dict_cache['i'] = [py_str+x]
        self.dict_cache['i'] = [py_str+x]
    else:
        return '<<<Success>>>:\n\n'+x
    
roys.create(
    agents = {
        'Coder': 'c = generate(i)',
        'Proxy': '_ = protocol(c)',
        },
    tools = {'protocol': custom_switch})
roys.start(requests = {'i': 'Compare the year-to-date gain for META and TESLA.'})

Emergent Multi-Agent Dynamics

Roy aims to facilitate the emergence of complex, adaptive multi-agent systems. It draws inspiration from biological and AI concepts to enable decentralized coordination and continual learning.

Survival of the Fittest - Periodically evaluate and selectively retain high-performing agents based on accuracy, speed etc. Agents adapt through peer interactions.
Mixture of Experts - Designate agent expertise, dynamically assemble specialist teams, and route tasks to optimal experts. Continuously refine and augment experts.

These mechanisms facilitate the emergence of capable, adaptive, and efficient agent collectives.

Get Involved

Roy is under active development. We welcome contributions - feel free to open issues and PRs!

Support the Project

If you found this project helpful or interesting and want to support more of these experiments, feel free to buy me a coffee!

Assets 2

10 Oct 15:48

JosefAlbers

v0.0.1-rc.1

dbb0fef

v0.0.1-rc.1 Pre-release

Pre-release

Roy

Roy is a lightweight alternative to autogen, for developing advanced multi-agent systems using language models. It redefines the paradigm of LLM application development, emphasizing simplicity, versatility, and transparency.

Features

Model-agnostic: Roy can be used with any language model, eliminating the need for external API keys. By default, it employs the 4-bit quantized wizard-coder-python, but you can swap it out for any LLM of your choice.
Modular and composable: Roy refines LLM interactions into modular, adaptable operations that can be used in isolation or composed in myriad ways to create sophisticated workflows.
Transparent: Opposed to frameworks that entangle operations behind multiple layers, Roy is transparent. Each method serves a distinct purpose, for instance, granting users complete oversight and control over the process.

Quickstart

git clone https://github.com/JosefAlbers/Roy
cd Roy
pip install -r requirements.txt
pip install -U transformers optimum accelerate auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/

from roy import Roy
roy = Roy()

Template-Based Generation with Constrained Beam Search

s = '"What date is today? Which big tech stock has the largest year-to-date gain this year? How much is the gain?\n'
s = roy.format(s)
roy.generate(s)
roy.generate(s, ('```python', '```'))                    # Generate a python code block
roy.generate(s, (('```python', '```javascript'), '```')) # Generate python or javascript codes
roy.generate(s, ('```python', 100, '```'))               # Generate a code block of size less than 100 tokens

Retrieval Augmented Generation

Agents can retrieve and leverage relevant information from external sources, enhancing their knowledge and capabilities.

s = 'Create a text to image generator.\n'
r = roy.retrieve(s, n_topk=3, src='huggingface')
[roy.generate(s) for s in r]

Auto-Feedback

Agents autonomously generate and execute feedback on each other's outputs, leading to continuous improvement and refinement.

s = "Create a secure and unique secret code word with a Python script that involves multiple steps to ensure the highest level of confidentiality and protection.\n"
for i in range(2):
    c = roy.generate(s)
    s += roy.execute(c)

Auto-Grinding

user_request = "Compare the year-to-date gain for META and TESLA.\n"
ai_response = roy.generate(user_request)
for i in range(2):
    shell_execution = roy.execute(ai_response)
    if 'ModuleNotFoundError' in shell_execution:
        roy.execute(roy.generate(roy.format(f'Write a shell command to address the error encountered while running this Python code:\n\n{shell_execution}')), False)
    elif 'Error' in shell_execution:
        ai_response = roy.generate(roy.format(f'Modify the code to address the error encountered:\n\n{shell_execution}'))
    else:
        break

Multi-Agent

Roy supports the creation of dynamic multi-agent systems:

from roy import Roys
roys = Roys()

# AutoFeedback
roys.create(agents = {'Coder': 'i = execute(generate(i))'})
roys.start(requests = {'i': 'Create a mobile application that can track the health of elderly people living alone in rural areas.\n'})

# Retrieval Augmented Generation
roys.create(
    agents = {
        'Retriever': 'r = retrieve(i)',
        'Generator': 'o = generate(r)',
        })
roys.start(requests = {'i': 'Create a Deutsch to English translator.\n'})

# Providing a custom tool to one of the agents using lambda
roys.create(
    agents = {
        'Coder': 'c = generate(i)',
        'Proxy': 'c = custom(execute(c))',
        },
    tools = {'custom': lambda x:f'Modify the code to address the error encountered:\n\n{x}' if 'Error' in x else None})
roys.start(requests = {'i': 'Compare the year-to-date gain for META and TESLA.\n'})

# Another way to create a custom tool for agents
def custom_switch(c):
    py_str = 'Modify the code to address the error encountered:\n\n'
    sh_str = 'Write a shell command to address the error encountered while running this Python code:\n\n'
    x = roys.execute(c)
    if 'ModuleNotFoundError' in x:
        roys.execute(roys.generate(sh_str+x))
        roys.dict_cache['c'] = [c]
    elif 'Error' in x:
        roys.dict_cache['i'] = [py_str+x]
    return 'Success:\n\n'+x
    
roys.create(
    agents = {
        'Coder': 'c = generate(i)',
        'Proxy': '_ = protocol(c)',
        },
    tools = {'protocol': custom_switch})
roys.start(requests = {'i': 'Compare the year-to-date gain for META and TESLA.\n'})

Self-Organizing Multi-Agent System

Envision a dynamic group chat where agents, each bearing distinct roles and defined by its unique constraints, affinities, and behaviors, converge to collaborate towards a predefined objective. This opens up a realm of possibilities, from brainstorming sessions to problem-solving think tanks. Drawing inspiration from biology and machine learning, Roy aspires to pioneer a perpetually evolving multi-agent environment.

Survival of the Fittest

Guided by the tenets of natural selection, this principle ensures the survival of only the most adept agents:

Fitness Metrics: Evaluate agents based on information quality, response speed, relevance, and feedback.
Agent Evaluation: Periodically assess agent performance.
Agent Reproduction: Allow top-performing agents to spawn new ones with similar attributes.
Agent Removal: Phase out underperformers to uphold quality.
Adaptive Learning: Agents evolve through their interactions.
Dynamic Environment: The chat setting adapts, pushing agents to be resilient and versatile.

Mixture of Experts

This principle promotes efficiency by allocating tasks to the most qualified agents:

Expertise Definition: Designate areas of expertise for each agent.
Dynamic Collaboration: Align relevant agents to address specific tasks.
Expert Evaluation: Constantly gauge the performance of expert agents.
Expert Refinement: Retrain, adjust, or replace underperforming experts.
Learning from Peers: Agents expand their knowledge horizons by learning from others.
Task Routing: Route tasks to the best-suited experts.
Layered Expertise: Deploy coordinators to guide collaborations for intricate tasks.
Human Expert Integration: Infuse human knowledge into the chat, amplifying collective intelligence.

By utilizing these principles, our self-organizing chat group remains in a state of perpetual evolution, always questing for the zenith of outcomes and ongoing enhancement.

Conclusion

Roy redefines the paradigm of LLM application development, emphasizing simplicity, versatility, and transparency. Whether your aim is a basic LLM interaction or an elaborate multi-agent system, Roy provides the architecture to realize it.

For those who've made it this far, I believe you share my passion. Dive into Roy, discover its potential, and join me in exploring the future of LLM applications.

If you found this project helpful or interesting and want to support more of these experiments, feel free to buy me a coffee!

Assets 2

08 Oct 19:50

JosefAlbers

v0.0.1-beta

bb87a01

v0.0.1-beta Pre-release

Pre-release

Roy

Roy is a lightweight alternative to autogen, for crafting advanced multi-agent systems using language models.

Features

Flexibility: Roy is model-agnostic, eliminating the need for external API keys. By default, it employs the 4-bit quantized wizard-coder-python, but you can swap it out for any LLM of your choice.
Composability: Roy refines the LLM interactions into modular, adaptable operations, each of which can be used in isolation or composed in myriad ways to create sophisticated workflows.
Clarity: Opposed to frameworks that entangle operations behind multiple layers, Roy is transparent. Each method serves a distinct purpose, for instance, granting users complete oversight and control over the process.

Quickstart

git clone https://github.com/JosefAlbers/Roy
cd Roy
pip install -r requirements.txt
pip install -U transformers optimum accelerate auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/

from roy import Roy
roy = Roy()

Template-Based Generation

s = '"What date is today? Which big tech stock has the largest year-to-date gain this year? How much is the gain?\n'
s = roy.format(s)
roy.generate(s)

Constrained Generation

roy.generate(s, ('```python', '```'))                    # Generate a python code block
roy.generate(s, (('```python', '```javascript'), '```')) # Generate python or javascript codes
roy.generate(s, ('```python', 100, '```'))               # Generate a code block of size less than 100 tokens

Retrieval Augmented Generation

s = 'Create a text to image generator.\n'
r = roy.retrieve(s, n_topk=3, src='huggingface')
r = roy.format('Modify the [Example Code] to fulfill the [User Request] using minimal changes. Keep the modifications minimal by making only the necessary modifications.\n\n[User Request]:\n"{user_request}"\n\n[Context]:\n{retrieved_docstr}\n\n[Example Code]:\n```python\n{retrieved_code}\n```', r)
[roy.generate(s) for s in r]

Auto Feedback

s = "Create a secure and unique secret code word with a Python script that involves multiple steps to ensure the highest level of confidentiality and protection.\n"

for i in range(3):
    c = roy.generate(s)
    s += roy.execute(c)

Auto Grind

user_request = "Compare the year-to-date gain for META and TESLA.\n"
ai_response = roy.generate(user_request)
for i in range(3):
    shell_execution = roy.execute(ai_response)
    if 'ModuleNotFoundError' in shell_execution:
        roy.execute(roy.generate(roy.format(f'Write a shell command to address the error encountered while running this Python code:\n\n{shell_execution}')), False)
    elif 'Error' in shell_execution:
        ai_response = roy.generate(roy.format(f'Modify the code to address the error encountered:\n\n{shell_execution}'))
    else:
        break

Self-Organizing Multi-Agent System

Envision a dynamic group chat where agents, each bearing distinct roles and defined by its unique constraints, affinities, and behaviors, converge to collaborates towards a predefined objective. This opens up a realm of possibilities, from brainstorming sessions to problem-solving think tanks. Drawing inspiration from biology and machine learning, Roy aspires to pioneer a perpetually evolving multi-agent environment.

Survival of the Fittest

Guided by the tenets of natural selection, this principle ensures the survival of only the most adept agents:

Fitness Metrics: Evaluate agents based on information quality, response speed, relevance, and feedback.
Agent Evaluation: Periodically assess agent performance.
Agent Reproduction: Allow top-performing agents to spawn new ones with similar attributes.
Agent Removal: Phase out underperformers to uphold quality.
Adaptive Learning: Agents evolve through their interactions.
Dynamic Environment: The chat setting adapts, pushing agents to be resilient and versatile.

Mixture of Experts

This principle promotes efficiency by allocating tasks to the most qualified agents:

Expertise Definition: Designate areas of expertise for each agent.
Dynamic Collaboration: Align relevant agents to address specific tasks.
Expert Evaluation: Constantly gauge the performance of expert agents.
Expert Refinement: Retrain, adjust, or replace underperforming experts.
Learning from Peers: Agents expand their knowledge horizons by learning from others.
Task Routing: Route tasks to the best-suited experts.
Layered Expertise: Deploy coordinators to guide collaborations for intricate tasks.
Human Expert Integration: Infuse human knowledge into the chat, amplifying collective intelligence.

By utilizing these principles, our self-organizing chat group remains in a state of perpetual evolution, always questing for the zenith of outcomes and ongoing enhancement.

Conclusion

For those who've made it this far, I believe you share my passion. Dive into Roy, discover its potential, and join me in exploring the future of LLM applications.

If you found this project helpful or interesting and want to support more of these experiments, feel free to buy me a coffee!

Assets 2

03 Oct 14:12

JosefAlbers

v0.0.1-alpha

b847778

v0.0.1-alpha Pre-release

Pre-release

Roy

Roy is a lightweight alternative to autogen, for crafting advanced multi-agent systems using language models.

Features

Flexibility: Roy is model-agnostic, eliminating the need for external API keys. By default, it employs the 4-bit quantized wizard-coder-python, but you can swap it out for any LLM of your choice.
Composability: Roy refines the LLM interactions into modular, adaptable operations, each of which can be used in isolation or composed in myriad ways to create sophisticated workflows.
Clarity: Opposed to frameworks that entangle operations behind multiple layers, Roy is transparent. Each method serves a distinct purpose, for instance, granting users complete oversight and control over the process.

Quickstart

git clone https://github.com/JosefAlbers/Roy
cd Roy
pip install -r requirements.txt

from roy import Roy
roy = Roy()

Template-Based Generation

s = 'Compare the year-to-date gain for META and TESLA.'
s = roy.format(s)
s = roy.generate(s)

Retrieval Augmented Generation

s = 'Create a text to image generator.'
r = roy.retrieve(s, n_topk=3, src='huggingface')
r = roy.format('Modify the [Example Code] to fulfill the [User Request] using minimal changes...', r)
[roy.generate(s) for s in r]

Auto Feedback

s = 'Find arxiv papers that show how are people studying trust calibration in AI based systems'
for i in range(3):
    c = roy.generate(s)
    s += c
    s += roy.execute(c)

Auto Grind

def auto_grind(user_request):
    cache = {'user_request': user_request, 'py_code': roy.generate(user_request)}
    for i in range(3):
        cache['sh_out'] = roy.execute(cache['py_code'])
        if 'Error' in cache['sh_out']:
            feedback = roy.format('Debug the Code ("script.py") that had been written for this problem: "{user_request}"\n\n[Code]:\n```python\n{py_code}\n```\n\n[Error]:\n{sh_out}', cache)
            cache['py_code'] = roy.generate(feedback)
        else:
            break
    return cache

auto_grind("Plot a chart of TESLA's stock price change YTD and save to 'stock_price_ytd.png'.")

Self-Organizing Multi-Agent System

Survival of the Fittest

Guided by the tenets of natural selection, this principle ensures the survival of only the most adept agents:

Fitness Metrics: Evaluate agents based on information quality, response speed, relevance, and feedback.
Agent Evaluation: Periodically assess agent performance.
Agent Reproduction: Allow top-performing agents to spawn new ones with similar attributes.
Agent Removal: Phase out underperformers to uphold quality.
Adaptive Learning: Agents evolve through their interactions.
Dynamic Environment: The chat setting adapts, pushing agents to be resilient and versatile.

Mixture of Experts

This principle promotes efficiency by allocating tasks to the most qualified agents:

Expertise Definition: Designate areas of expertise for each agent.
Dynamic Collaboration: Align relevant agents to address specific tasks.
Expert Evaluation: Constantly gauge the performance of expert agents.
Expert Refinement: Retrain, adjust, or replace underperforming experts.
Learning from Peers: Agents expand their knowledge horizons by learning from others.
Task Routing: Route tasks to the best-suited experts.
Layered Expertise: Deploy coordinators to guide collaborations for intricate tasks.
Human Expert Integration: Infuse human knowledge into the chat, amplifying collective intelligence.

By utilizing these principles, our self-organizing chat group remains in a state of perpetual evolution, always questing for the zenith of outcomes and ongoing enhancement.

Conclusion

For those who've made it this far, we believe you share our passion. Dive into Roy, discover its potential, and collaborate with us to sculpt the future of LLM applications.

Assets 2

Releases: JosefAlbers/Roy

Rapid Prototyping of Agents with Hotswappable Components

Roy

Features

Quickstart

Rapid Benchmarking

Constrained Beam Search

Retrieval Augmented Generation

Auto-Feedback

Auto-Grinding

Multi-Agent

Emergent Multi-Agent Dynamics

Get Involved

Support the Project

v0.0.1-rc.1

Roy

Features

Quickstart

Template-Based Generation with Constrained Beam Search

Retrieval Augmented Generation

Auto-Feedback

Auto-Grinding

Multi-Agent

Self-Organizing Multi-Agent System

Survival of the Fittest

Mixture of Experts

Conclusion

v0.0.1-beta

Roy

Features

Quickstart

Template-Based Generation

Constrained Generation

Retrieval Augmented Generation

Auto Feedback

Auto Grind

Self-Organizing Multi-Agent System

Survival of the Fittest

Mixture of Experts

Conclusion

v0.0.1-alpha

Roy

Features

Quickstart

Template-Based Generation

Retrieval Augmented Generation

Auto Feedback

Auto Grind

Self-Organizing Multi-Agent System

Survival of the Fittest

Mixture of Experts

Conclusion