A Meteor package that allows you to attach a schema to a Mongo.Collection. Automatically validates against that schema when inserting and updating from client or server code.
This package requires the simpl-schema NPM package, which defines the schema syntax and provides the validation logic.
Table of Contents generated with DocToc
- Installation
- Why Use Collection2
- Attaching a Schema to a Collection
- Schema Format
- Passing Options
- Validation Contexts
- Validating Without Inserting or Updating
- Inserting or Updating Without Validating
- Inserting or Updating Without Cleaning
- Inserting or Updating Bypassing Collection2 Entirely
- Additional SimpleSchema Options
- What Happens When The Document Is Invalid?
- More Details
- Community Add-On Packages
- Problems
- Contributing
- Publishing a New Release to Atmosphere
- Contributors
- Backers
- Sponsors
In your Meteor app directory, enter:
meteor add aldeed:[email protected]
meteor npm install --save simpl-schema- While adding allow/deny rules ensures that only authorized users can edit a document from the client, adding a schema ensures that only acceptable properties and values can be set within that document from the client. Thus, client side inserts and updates can be allowed without compromising security or data integrity.
- Schema validation for all inserts and updates is reactive, allowing you to easily display customizable validation error messages to the user without any event handling.
- Schema validation for all inserts and updates is automatic on both the client and the server, providing both speed and security.
- The aldeed:autoform package can take your collection's schema and automatically create HTML5 forms based on it. AutoForm provides automatic database operations, method calls, validation, and user interface reactivity. You have to write very little markup and no event handling. Refer to the AutoForm documentation for more information.
Let's say we have a normal "books" collection, defined in code that runs on both client and server (common.js):
const Books = new Mongo.Collection("books");Let's create a SimpleSchema schema for this collection. We'll do this in common.js, too:
const Schemas = {};
Schemas.Book = new SimpleSchema({
title: {
type: String,
label: "Title",
max: 200
},
author: {
type: String,
label: "Author"
},
copies: {
type: SimpleSchema.Integer,
label: "Number of copies",
min: 0
},
lastCheckedOut: {
type: Date,
label: "Last date this book was checked out",
optional: true
},
summary: {
type: String,
label: "Brief summary",
optional: true,
max: 1000
}
});Once we have the SimpleSchema instance, all we need to do is attach it to our collection using the attachSchema method. Again, we will do this in common.js:
Books.attachSchema(Schemas.Book);Now that our collection has a schema, we can do a validated insert on either the client or the server:
Books.insert({title: "Ulysses", author: "James Joyce"}, (error, result) => {
//The insert will fail, error will be set,
//and result will be undefined or false because "copies" is required.
//
//The list of errors is available on `error.invalidKeys` or by calling Books.simpleSchema().namedContext().validationErrors()
});Or we can do a validated update:
Books.update(book._id, {$unset: {copies: 1}}, (error, result) => {
//The update will fail, error will be set,
//and result will be undefined or false because "copies" is required.
//
//The list of errors is available on `error.invalidKeys` or by calling Books.simpleSchema().namedContext().validationErrors()
});Normally, if call attachSchema multiple times, the schemas are merged. If you use the replace: true option, then it will replace the previously attached schema. However, in some cases you might actually want both schemas attached, with different documents validated against different schemas.
Here is an example:
Products.attachSchema(SimpleProductSchema, {selector: {type: 'simple'}});
Products.attachSchema(VariantProductSchema, {selector: {type: 'variant'}});Now both schemas are attached. When you insert a document where type: 'simple' in the document, it will validate against only the SimpleProductSchema. When you insert a document where type: 'variant' in the document, it will validate against only the VariantProductSchema.
Alternatively, you can pass a selector option when inserting to choose which schema to use:
Products.insert({ title: 'This is a product' }, { selector: { type: 'simple' } });For an update or upsert, the matching selector can be in the query, the modifier $set object, or the selector option.
If your validation requires that your doc be transformed using the collection's transform function prior to being validated, then you must pass the transform: true option to attachSchema when you attach the schema:
Books.attachSchema(Schemas.Book, {transform: true});By default, if a collection already has a schema attached, attachSchema will combine the new schema with the existing. Pass the replace: true option to attachSchema to discard any existing schema.
Obviously, when you attach a schema, you must know what the schema should be. For Meteor.users,
here is an example schema, which you might have to adjust for your own needs:
const Schema = {};
Schema.UserCountry = new SimpleSchema({
name: {
type: String
},
code: {
type: String,
regEx: /^[A-Z]{2}$/
}
});
Schema.UserProfile = new SimpleSchema({
firstName: {
type: String,
optional: true
},
lastName: {
type: String,
optional: true
},
birthday: {
type: Date,
optional: true
},
gender: {
type: String,
allowedValues: ['Male', 'Female'],
optional: true
},
organization : {
type: String,
optional: true
},
website: {
type: String,
regEx: SimpleSchema.RegEx.Url,
optional: true
},
bio: {
type: String,
optional: true
},
country: {
type: Schema.UserCountry,
optional: true
}
});
Schema.User = new SimpleSchema({
username: {
type: String,
// For accounts-password, either emails or username is required, but not both. It is OK to make this
// optional here because the accounts-password package does its own validation.
// Third-party login packages may not require either. Adjust this schema as necessary for your usage.
optional: true
},
emails: {
type: Array,
// For accounts-password, either emails or username is required, but not both. It is OK to make this
// optional here because the accounts-password package does its own validation.
// Third-party login packages may not require either. Adjust this schema as necessary for your usage.
optional: true
},
"emails.$": {
type: Object
},
"emails.$.address": {
type: String,
regEx: SimpleSchema.RegEx.Email
},
"emails.$.verified": {
type: Boolean
},
// Use this registered_emails field if you are using splendido:meteor-accounts-emails-field / splendido:meteor-accounts-meld
registered_emails: {
type: Array,
optional: true
},
'registered_emails.$': {
type: Object,
blackbox: true
},
createdAt: {
type: Date
},
profile: {
type: Schema.UserProfile,
optional: true
},
// Make sure this services field is in your schema if you're using any of the accounts packages
services: {
type: Object,
optional: true,
blackbox: true
},
// DISCLAIMER: This only applies to the first and second version of meteor-roles package.
// https://github.com/Meteor-Community-Packages/meteor-collection2/issues/399
// Add `roles` to your schema if you use the meteor-roles package.
// Option 1: Object type
// If you specify that type as Object, you must also specify the
// `Roles.GLOBAL_GROUP` group whenever you add a user to a role.
// Example:
// Roles.addUsersToRoles(userId, ["admin"], Roles.GLOBAL_GROUP);
// You can't mix and match adding with and without a group since
// you will fail validation in some cases.
roles: {
type: Object,
optional: true,
blackbox: true
},
// Option 2: [String] type
// If you are sure you will never need to use role groups, then
// you can specify [String] as the type
roles: {
type: Array,
optional: true
},
'roles.$': {
type: String
},
// In order to avoid an 'Exception in setInterval callback' from Meteor
heartbeat: {
type: Date,
optional: true
}
});
Meteor.users.attachSchema(Schema.User);This schema has not been thoroughly vetted to ensure that it accounts for all possible properties the accounts packages might try to set. Furthermore, any other packages you add might also try to set additional properties. If you see warnings in the console about keys being removed, that's a good indication that you should add those keys to the schema.
Note also that this schema uses the blackbox: true option for simplicity. You might choose instead
to figure out a more specific schema.
(If you figure out a more accurate Meteor.users schema, documentation pull requests are welcome.)
Refer to the simpl-schema package documentation for a list of all the available schema rules and validation methods.
Use the MyCollection.simpleSchema() method to access the attached SimpleSchema
instance for a Mongo.Collection instance. For example:
MyCollection.simpleSchema().validate(doc);In Meteor, the update function accepts an options argument. Collection2 changes the insert function signature to also accept options in the same way, as an optional second argument. Whenever this documentation says to "use X option", it's referring to this options argument. For example:
myCollection.insert(doc, {validate: false});In the examples above, note that we called namedContext() with no arguments
to access the SimpleSchema reactive validation methods. Contexts let you keep
multiple separate lists of invalid keys for a single collection.
In practice you might be able to get away with always using the default context.
It depends on what you're doing. If you're using the context's reactive methods
to update UI elements, you might find the need to use multiple contexts. For example,
you might want one context for inserts and one for updates, or you might want
a different context for each form on a page.
To use a specific named validation context, use the validationContext option
when calling insert or update:
Books.insert({title: "Ulysses", author: "James Joyce"}, { validationContext: "insertForm" }, (error, result) => {
//The list of errors is available by calling Books.simpleSchema().namedContext("insertForm").validationErrors()
});
Books.update(book._id, {$unset: {copies: 1}}, { validationContext: "updateForm" }, (error, result) => {
//The list of errors is available by calling Books.simpleSchema().namedContext("updateForm").validationErrors()
});It's also possible to validate a document without performing the actual insert or update:
Books.simpleSchema().namedContext().validate({title: "Ulysses", author: "James Joyce"}, {modifier: false});Set the modifier option to true if the document is a mongo modifier object.
You can also validate just one key in the document:
Books.simpleSchema().namedContext().validate({title: "Ulysses", author: "James Joyce"}, {modifier: false, keys: ['title']});Or you can specify a certain validation context when calling either method:
Books.simpleSchema().namedContext("insertForm").validate({title: "Ulysses", author: "James Joyce"}, {modifier: false});
Books.simpleSchema().namedContext("insertForm").validate({title: "Ulysses", author: "James Joyce"}, {modifier: false, keys: ['title']});Refer to the simpl-schema package documentation for more information about these methods.
To skip validation, use the validate: false option when calling insert or
update. On the client (untrusted code), this will skip only client-side
validation. On the server (trusted code), it will skip all validation. The object is still cleaned and autoValues are still generated.
To skip object property filtering, set the filter option to false when you call insert or update.
To skip automatic value conversion, set the autoConvert option to false when you call insert or update.
To skip removing empty strings, set the removeEmptyStrings option to false when you call insert or update.
To skip adding automatic values, set the getAutoValues option to false when you call insert or update. This works only in server code.
To pick or omit fields from the schema for the operation, set the 'pick' or 'omit' option respectively to an array of schema field names. These options are mutually exclusive, so you cannot have both present in the options object at the same time.
This is the implementation of pick and omit functionality from simple-schema, but on your DB calls like this:
// Will insert everything except 'noop'
collection.insert(
{ foo: 'foo', noop: 'nooooo', bar: 'whiskey' },
{ omit: ['noop'] });// Pick only 'foo'
collection.update(
{ _id: 'myid' },
{ $set: { foo: 'test', bar: 'changed' } },
{ pick: ['foo'] });Even if you skip all validation and cleaning, Collection2 will still do some object parsing that can take a long time for a large document. To bypass this, set the bypassCollection2 option to true when you call insert or update. This works only in server code.
In addition to all the other schema validation options documented in the simpl-schema package, the collection2 package adds additional options explained in this section.
See https://github.com/aldeed/meteor-schema-index
See https://github.com/aldeed/meteor-schema-deny
The autoValue option is provided by the SimpleSchema package and is documented
there. Collection2 adds the following properties to this for any autoValue
function that is called as part of a C2 database operation:
- isInsert: True if it's an insert operation
- isUpdate: True if it's an update operation
- isUpsert: True if it's an upsert operation (either
upsert()orupsert: true) - userId: The ID of the currently logged in user. (Always
nullfor server-initiated actions.) - isFromTrustedCode: True if the insert, update, or upsert was initiated from trusted (server) code
- docId: The
_idproperty of the document being inserted or updated. For an insert, this will be set only when it is provided in the insert doc, or when the operation is initiated on the client. For an update or upsert, this will be set only when the selector is or includes the_id, or when the operation is initiated on the client.
Note that autoValue functions are run on the client only for validation purposes, but the actual value saved will always be generated on the server, regardless of whether the insert/update is initiated from the client or from the server.
There are many possible use cases for autoValue. It's probably easiest to
explain by way of several examples:
{
// Force value to be current date (on server) upon insert
// and prevent updates thereafter.
createdAt: {
type: Date,
autoValue: function() {
if (this.isInsert) {
return new Date();
} else if (this.isUpsert) {
return {$setOnInsert: new Date()};
} else {
this.unset(); // Prevent user from supplying their own value
}
}
},
// Force value to be current date (on server) upon update
// and don't allow it to be set upon insert.
updatedAt: {
type: Date,
autoValue: function() {
if (this.isUpdate) {
return new Date();
}
},
denyInsert: true,
optional: true
},
// Whenever the "content" field is updated, automatically store
// the first word of the content into the "firstWord" field.
firstWord: {
type: String,
optional: true,
autoValue: function() {
var content = this.field("content");
if (content.isSet) {
return content.value.split(' ')[0];
} else {
this.unset();
}
}
},
// Whenever the "content" field is updated, automatically
// update a history array.
updatesHistory: {
type: Array,
optional: true,
autoValue: function() {
var content = this.field("content");
if (content.isSet) {
if (this.isInsert) {
return [{
date: new Date(),
content: content.value
}];
} else {
return {
$push: {
date: new Date,
content: content.value
}
};
}
} else {
this.unset();
}
}
},
'updatesHistory.$': {
type: Object,
},
'updatesHistory.$.date': {
type: Date,
optional: true
},
'updatesHistory.$.content': {
type: String,
optional: true
},
// Automatically set HTML content based on markdown content
// whenever the markdown content is set.
htmlContent: {
type: String,
optional: true,
autoValue: function(doc) {
var markdownContent = this.field("markdownContent");
if (Meteor.isServer && markdownContent.isSet) {
return MarkdownToHTML(markdownContent.value);
}
}
}
}The custom option is provided by the SimpleSchema package and is documented
there. Collection2 adds the following properties to this for any custom
function that is called as part of a C2 database operation:
- isInsert: True if it's an insert operation
- isUpdate: True if it's an update operation
- isUpsert: True if it's an upsert operation (either
upsert()orupsert: true) - userId: The ID of the currently logged in user. (Always
nullfor server-initiated actions.) - isFromTrustedCode: True if the insert, update, or upsert was initiated from trusted (server) code
- docId: The
_idproperty of the document being inserted or updated. For an insert, this will be set only when it is provided in the insert doc, or when the operation is initiated on the client. For an update or upsert, this will be set only when the selector is or includes the_id, or when the operation is initiated on the client.
The callback you specify as the last argument of your insert() or update() call will have the first argument (error) set to an Error instance. The error message for the first invalid key is set in the error.message, and the full validationErrors array is available on error.invalidKeys. This is true on both client and server, even if validation for a client-initiated operation does not fail until checked on the server.
If you attempt a synchronous operation in server code, the same validation error is thrown since there is no callback to pass it to. If this happens in a server method (defined with Meteor.methods), a more generic Meteor.Error is passed to your callback back on the client. This error does not have an invalidKeys property, but it does have the error message for the first invalid key set in error.reason.
Generally speaking, you would probably not use the Error for displaying to the user. You can instead use the reactive methods provided by the SimpleSchema validation context to display the specific error messages to the user somewhere in the UI. The autoform package provides some UI components and helpers for this purpose.
For the curious, this is exactly what Collection2 does before every insert or update:
- Removes properties from your document or mongo modifier object if they are not explicitly listed in the schema. (To skip this, set the
filteroption tofalsewhen you callinsertorupdate.) - Automatically converts some properties to match what the schema expects, if possible. (To skip this, set the
autoConvertoption tofalsewhen you callinsertorupdate.) - Optimizes your operation so that empty string values will not be stored. (To skip this, set the
removeEmptyStringsoption tofalsewhen you callinsertorupdate.) - Adds automatic (forced or default) values based on your schema. Values are added only on the server and will make their way back to your client when your subscription is updated. (To skip this in server code, set the
getAutoValuesoption tofalsewhen you callinsertorupdate.) - Validates your document or mongo modifier object. (To skip this, set the
validateoption tofalsewhen you callinsertorupdate.) - Performs the insert or update like normal, only if it was valid.
Collection2 is simply calling SimpleSchema methods to do these things. The validation happens on both the client and the server for client-initiated actions, giving you the speed of client-side validation along with the security of server-side validation.
The davidyaha:collection2-migrations package can watch for schema changes between server restarts and perform some automatic data migration and cleanup.
You might find yourself in a situation where it seems as though validation is not working correctly. First, you should enable SimpleSchema debug mode by setting SimpleSchema.debug = true, which may log some additional information. If you're still confused, read through the following tricky, confusing situations.
One critical thing to know about Collection2 and SimpleSchema is that they don't validate the saved document but rather the proposed insert doc or the update modifier. In the case of updates, this means there is some information unknown to SimpleSchema, such as whether the array object you're attempting to modify already exists or not. If it doesn't exist, MongoDB would create it, so SimpleSchema will validate conservatively. It will assume that any properties not set by the modifier will not exist after the update. This means that the modifier will be deemed invalid if any required keys in the same object are not explicitly set in the update modifier.
For example, say we add the following keys to our "books" schema:
{
borrowedBy: {
type: Array
},
'borrowedBy.$': {
type: Object
},
"borrowedBy.$.name": {
type: String
},
"borrowedBy.$.email": {
type: String,
regEx: SimpleSchema.RegEx.Email
},
}Every object in the borrowedBy array must have a name and email property.
Now we discover that the name is incorrect in item 1, although the email address is correct. So we will just set the name to the correct value:
Books.update(id, {$set: {"borrowedBy.1.name": "Frank"}});However, this will not pass validation. Why? Because we don't know whether item 1 in the borrowedBy array already exists, so we don't know whether it will have the required email property after the update finishes.
There are three ways to make this work:
$setthe entire object$setall required keys in the object- Perform the update on the server, and pass the
validate: falseoption to skip validation.
When this situation occurs on the client with an autoForm, it generally does not cause any problems because AutoForm is smart enough to $set the entire object; it's aware of this potential issue. However, this means that you need to ensure that all required properties are represented by an input on the form. In our example, if you want an autoForm that only shows a field for changing the borrowedBy name and not the email, you should include both fields but make the email field hidden. Alternatively, you can submit the autoForm to a server method and then do a server update without validation.
Although these examples focused on an array of objects, sub-objects are treated basically the same way.
Anyone is welcome to contribute. Fork, make and test your changes (meteor test-packages ./),
and then submit a pull request.
cd tests
meteor npm i && npm testcd tests
meteor npm i && npm run test:watchCheck out master branch.
In /package/collection2/package.js, increment the version according to semantic versioning rules.
In CHANGELOG.md, add a heading for this version and a description of changes committed since the previous version.
Verify that docs in README.md are updated for these changes.
In root of project, run doctoc .. This updates both TOCs in the markdown files.
Run tests (see "Running Tests" section above).
cd to package/collection2 directory and run meteor publish. (You must have permission.)
Commit all version and docs changes, tag, and push:
git add .
git commit -m "publish 1.2.3"
git push origin master
git tag 1.2.3 && git push --tags(substitute actual version number)
This project exists thanks to all the people who contribute. [Contribute].
@mquandalle
(Add yourself if you should be listed here.)
Thank you to all our backers! 🙏 [Become a backer]
Support this project by becoming a sponsor. Your logo will show up here with a link to your website. [Become a sponsor]
