Skip to content

Commit

Permalink
Merge pull request #318 from vespa-engine/kkraune/nodebooks-deploy
Browse files Browse the repository at this point in the history 
Kkraune/notebooks deploy MERGEOK
  • Loading branch information
@thigm85
thigm85 authored Apr 21, 2022
2 parents 8719ecf + ace97ca commit 90d9439
Show file tree
Hide file tree
Showing 12 changed files with 291 additions and 456 deletions.
136 changes: 77 additions & 59 deletions docs/sphinx/source/deploy-docker.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,26 @@
"source": [
"# Deploy with Docker \n",
"\n",
"The simplest way to deploy a Vespa app."
"The simplest way to deploy a Vespa app.\n",
"\n",
"Install `pyvespa` and make sure Docker is running with minimum 4G:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Total Memory: 11.7GiB\r\n"
]
}
],
"source": [
"!docker info | grep \"Total Memory\""
]
},
{
Expand Down Expand Up @@ -67,15 +86,8 @@
"import os\n",
"from vespa.deployment import VespaDocker\n",
"\n",
"disk_folder = os.path.join(os.getenv(\"WORK_DIR\"), \"sample_application\") # specify your desired absolute path here\n",
"vespa_docker = VespaDocker(\n",
" port=8080,\n",
" disk_folder=disk_folder\n",
")\n",
"\n",
"app = vespa_docker.deploy(\n",
" application_package = app_package,\n",
")"
"vespa_docker = VespaDocker()\n",
"app = vespa_docker.deploy(application_package=app_package)"
]
},
{
Expand All @@ -91,11 +103,8 @@
"metadata": {},
"outputs": [],
"source": [
"from shutil import rmtree\n",
"\n",
"vespa_docker.container.stop()\n",
"vespa_docker.container.remove()\n",
"rmtree(disk_folder, ignore_errors=True)"
"vespa_docker.container.remove()"
]
},
{
Expand All @@ -109,14 +118,17 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"It is important to know that `pyvespa` simply provides a convenient API to define Vespa application packages from python. `vespa_docker.deploy` export Vespa configuration files to the `disk_folder` defined above. Going through those files is a nice way to start learning about Vespa syntax."
"It is important to know that `pyvespa` simply provides a convenient API\n",
"to define Vespa application packages from python.\n",
"Going through those files is a nice way to start learning about Vespa syntax."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It is also possible to export the Vespa configuration files representing an application package created with `pyvespa` without deploying the application by using the `export_application_package` method: "
"Use [to_files](reference-api.rst#vespa.package.ApplicationPackage.to_files)\n",
"to export the application package package created with `pyvespa` from code to files:"
]
},
{
Expand All @@ -125,17 +137,16 @@
"metadata": {},
"outputs": [],
"source": [
"vespa_docker.export_application_package(\n",
" application_package=app_package, \n",
")"
"from pathlib import Path\n",
"Path(\"mydir\").mkdir(parents=True, exist_ok=True)\n",
"app_package.to_files(\"mydir\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This will export the application files to an `application` folder within the `disk_folder`.\n",
"Remove the files after use:"
"Inspect the exported files:"
]
},
{
Expand All @@ -144,48 +155,60 @@
"metadata": {},
"outputs": [],
"source": [
"rmtree(disk_folder, ignore_errors=True)"
"!find mydir"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deploy application package from config files"
"It is also possible to export the application package as a zipped file\n",
"using [to_zipfile](reference-api.rst#vespa.package.ApplicationPackage.to_zipfile).\n",
"The zipfile can later be deployed with pyvespa or the [Vespa CLI](https://docs.vespa.ai/en/vespa-cli.html):"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"app_package.to_zipfile(\"myzip.zip\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `pyvespa` API provides a subset of the functionality available in Vespa. The reason is that `pyvespa` is meant to be used as an experimentation tool for Information Retrieval (IR) and not for building production-ready applications. So, the python API expands based on the needs to replicate common use cases that often require IR experimentation.\n",
"\n",
"If the application requires functionality or fine-tuning not available in `pyvespa`, simply build it directly using Vespa configuration files as shown in [many examples](https://docs.vespa.ai/en/getting-started.html) on Vespa docs. But even in this case, one can still get value out of `pyvespa` by deploying it from python based on the Vespa configuration files stored on disk.\n",
"\n",
"Make sure Docker is running with minimum 4G:"
"Remove the files after use:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Total Memory: 11.7GiB\r\n"
]
}
],
"outputs": [],
"source": [
"!docker info | grep \"Total Memory\""
"from shutil import rmtree\n",
"rmtree(\"mydir\", ignore_errors=True)\n",
"rmtree(\"myzip.zip\", ignore_errors=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deploy application package from config files"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `pyvespa` API provides a subset of the functionality available in Vespa. The reason is that `pyvespa` is meant to be used as an experimentation tool for Information Retrieval (IR) and not for building production-ready applications. So, the python API expands based on the needs to replicate common use cases that often require IR experimentation.\n",
"\n",
"If the application requires functionality or fine-tuning not available in `pyvespa`, simply build it directly using Vespa configuration files as shown in [many examples](https://docs.vespa.ai/en/getting-started.html) on Vespa docs. But even in this case, one can still get value out of `pyvespa` by deploying it from python based on the Vespa configuration files stored on disk.\n",
"\n",
"Clone and deploy the news search app covered in this [Vespa tutorial](https://docs.vespa.ai/en/tutorials/news-3-searching.html):"
]
},
Expand All @@ -199,17 +222,17 @@
"output_type": "stream",
"text": [
"Cloning into 'sample-apps'...\n",
"remote: Enumerating objects: 1138, done.\u001b[K\n",
"remote: Counting objects: 100% (1138/1138), done.\u001b[K\n",
"remote: Compressing objects: 100% (754/754), done.\u001b[K\n",
"remote: Total 1138 (delta 189), reused 889 (delta 109), pack-reused 0\u001b[K\n",
"remote: Enumerating objects: 1138, done.\u001B[K\n",
"remote: Counting objects: 100% (1138/1138), done.\u001B[K\n",
"remote: Compressing objects: 100% (754/754), done.\u001B[K\n",
"remote: Total 1138 (delta 189), reused 889 (delta 109), pack-reused 0\u001B[K\n",
"Receiving objects: 100% (1138/1138), 18.94 MiB | 9.38 MiB/s, done.\n",
"Resolving deltas: 100% (189/189), done.\n"
]
}
],
"source": [
"!git clone --depth 1 https://github.com/vespa-engine/sample-apps.git $WORK_DIR/sample-apps"
"!git clone --depth 1 https://github.com/vespa-engine/sample-apps.git sample-apps"
]
},
{
Expand All @@ -228,8 +251,8 @@
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[01;34msample-apps/news/app-3-searching/\u001b[00m\r\n",
"├── \u001b[01;34mschemas\u001b[00m\r\n",
"\u001B[01;34msample-apps/news/app-3-searching/\u001B[00m\r\n",
"├── \u001B[01;34mschemas\u001B[00m\r\n",
"│   └── news.sd\r\n",
"└── services.xml\r\n",
"\r\n",
Expand All @@ -245,8 +268,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Deploy to a Docker container from disk.\n",
"Note that `disk_folder` must be an absolute path, for Docker to bind the volume:"
"Deploy to a Docker container from disk:"
]
},
{
Expand All @@ -270,12 +292,10 @@
"import os\n",
"from vespa.deployment import VespaDocker\n",
"\n",
"disk_folder=os.getenv(\"WORK_DIR\") + \"/sample-apps/news/app-3-searching\"\n",
"vespa_docker_news = VespaDocker(\n",
" disk_folder=disk_folder,\n",
" port=8090\n",
")\n",
"app = vespa_docker_news.deploy_from_disk(application_name=\"news\")"
"vespa_docker_news = VespaDocker(port=8080)\n",
"app = vespa_docker_news.deploy_from_disk(\n",
" application_name=\"news\",\n",
" application_folder=\"sample-apps/news/app-3-searching\")"
]
},
{
Expand All @@ -291,11 +311,9 @@
"metadata": {},
"outputs": [],
"source": [
"from shutil import rmtree\n",
"\n",
"rmtree(os.getenv(\"WORK_DIR\") + \"/sample-apps\", ignore_errors=True)\n",
"vespa_docker_news.container.stop()\n",
"vespa_docker_news.container.remove()"
"vespa_docker_news.container.remove()\n",
"rmtree(\"sample-apps\", ignore_errors=True)"
]
}
],
Expand All @@ -320,4 +338,4 @@
},
"nbformat": 4,
"nbformat_minor": 2
}
}
11 changes: 2 additions & 9 deletions docs/sphinx/source/exchange-data-with-app.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,7 @@
"from vespa.gallery import QuestionAnswering\n",
"\n",
"app_package = QuestionAnswering()\n",
"\n",
"disk_folder = os.path.join(os.getenv(\"WORK_DIR\"), \"sample_application\")\n",
"vespa_docker = VespaDocker(\n",
" port=8081, \n",
" disk_folder=disk_folder # requires absolute path\n",
")\n",
"vespa_docker = VespaDocker()\n",
"app = vespa_docker.deploy(application_package=app_package)"
]
},
Expand Down Expand Up @@ -631,9 +626,7 @@
"outputs": [],
"source": [
"# this is a hidden cell. It will not show on the documentation HTML.\n",
"from shutil import rmtree\n",
"\n",
"rmtree(disk_folder, ignore_errors=True)\n",
"vespa_docker.container.stop()\n",
"vespa_docker.container.remove()"
]
Expand All @@ -660,4 +653,4 @@
},
"nbformat": 4,
"nbformat_minor": 5
}
}
Loading

0 comments on commit 90d9439

Please sign in to comment.