hiro/grapher
1
0
Fork 0
mirror of https://github.com/vale981/grapher synced 2025-03-04 09:01:40 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions

Added documentation

Browse source
This commit is contained in:
Theodor Diaconu 2017-11-30 22:11:43 +02:00
parent 324e2b78fd
commit 35f297c395
19 changed files with 3048 additions and 59 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
CHANGELOG.md
View file

@ -2,6 +2,8 @@
- Added link caching
- Added named query results caching
- Added subbody to NamedQuery
- Added named query first resolver
- Bug fixes and other small stuff
## 1.2.5
- Support for promises via .fetchSync and .fetchOneSync for client-side queries

7
MIGRATION.md
View file

@ -37,4 +37,9 @@ createNamedQuery('xxx', {});
// working
createQuery('xxx', {});
```
```
If you used `$postFilter` in your queries, rename it to `$postFilters`
If you used resolver links, migrate to reducers.

145
README.md
View file

@ -1,76 +1,105 @@
Welcome to Grapher
==================
# Grapher 1.3
[![Build Status](https://api.travis-ci.org/cult-of-coders/grapher.svg?branch=master)](https://travis-ci.org/cult-of-coders/grapher)
Documentation
-------------
[http://grapher.cultofcoders.com](http://grapher.cultofcoders.com/)
*Grapher* is a data retrieval layer inside Meteor and MongoDB.
Long Term Support
-----------------
Version 1.2 will be supported until 2020.
Main features:
- Innovative way to make MongoDB relational
- Reactive data graphs for high availability
- Incredible performance
- Denormalization Modules
- Connection to external data sources
- Usable from anywhere
What ?
------
*Grapher* is a high performance data fetcher and collection relationship manager for Meteor and MongoDB:
It marks a stepping stone into evolution of data, enabling developers to write complex and secure code,
while maintaining the code base easy to understand.
1. Makes data MongoDB denormalization easy (storing and linking data in different collections)
2. You can link your MongoDB data with any type of database, and fetch it via Queries
3. You have the same API for data-fetching whether you want your data to be reactive or not.
4. It is compatible with simpl-schema and the older version of it.
Sample
-------------
To give you an idea how this works, you can fetch the data like this:
```
{
users: {
profile: 1,
githubTickets: {},
posts: {
title: 1,
comments: {
text: 1,
date: 1,
author: {
profile: 1
}
}
}
}
}
```
Updates
-------
Check-out the [CHANGELOG](CHANGELOG.md) for latest updates.
Installation
------------
## Installation
```
meteor add cultofcoders:grapher
```
Useful packages and integrations
--------------------------------
## [Documentation](docs/table_of_contents.md)
#### Integration with React (cultofcoders:grapher-react)
## [API](docs/api.md)
Provides you with an easy to use "createQueryContainer" function.
## Quick Illustration
- [Atmosphere](https://atmospherejs.com/cultofcoders/grapher-react)
- [GitHub](https://github.com/cult-of-coders/grapher-react/)
<table>
<tr>
<td width="50%">
<pre>
import {createQuery} from 'meteor/cultofcoders-grapher';
#### Live View (cultofcoders:grapher-live)
createQuery({
posts: {
title: 1,
author: {
fullName: 1
},
comments: {
text: 1,
createdAt: 1,
author: {
fullName: 1
}
},
categories: {
name: 1
}
}
}).fetch();
</pre>
</td>
<td width="50%">
<pre>
[
{
_id: 'postId',
title: 'Introducing Grapher',
author: {
_id: 'authorId',
fullName: 'John Smith
},
comments: [
{
_id: 'commentId',
text: 'Nice article!,
createdAt: Date,
author: {
fullName: 1
}
}
],
categories: [ {_id: 'categoryId', name: 'JavaScript'} ]
}
]
</pre>
</td>
</tr>
</table>
## Useful packages and integrations
### Live View (cultofcoders:grapher-live)
Provides a playground for grapher and provides documentation of your data
- [Atmosphere](https://atmospherejs.com/cultofcoders/grapher-live)
- [GitHub](https://github.com/cult-of-coders/grapher-live)
https://github.com/cult-of-coders/grapher-live
Boiler plate Meteor + React + Grapher
-------------------------------------
https://github.com/cult-of-coders/grapher-boilerplate
### Integration with UI Frameworks (cultofcoders:grapher-react)
#### React
https://github.com/cult-of-coders/grapher-react
#### Vue JS
https://github.com/Herteby/grapher-vue
https://github.com/cult-of-coders/grapher-react
## Premium Support
If you are looking to integrate Grapher in your apps and want online or on-site consulting and training,
shoot us an e-mail contact@cultofcoders.com, we will be more than happy to aid you.

164
docs/api.md Normal file
View file

@ -0,0 +1,164 @@
# API
Use this as a cheatsheet after you have read the full documentation.
### Adding Links
```js
Collection.addLinks({
linkName: {
collection, // Mongo.Collection
type, // 'one' or 'many'
metadata, // Boolean
field, // String
denormalize: {
field, // String
body, // Body from related collection
}
}
})
Collection.addLinks({
linkName: {
collection, // Mongo.Collection
inversedBy, // The link name from the other side
denormalize: {
field, // String
body, // Body from related collection
}
}
})
```
### Adding Reducers
```js
Collection.addReducers({
reducerName: {
body, // Object, dependency graph
compute(object) {
// anything
}
}
})
```
### Creating Named Query
```js
Collection.createQuery('queryName', {
$options, // Mongo Options {sort, limit, skip}
$filters, // Mongo Filters
$filter({filters, options, params}) {}, // Function or [Function]
$postOptions, // {limit, sort, skip}
$postFilters, // any sift() available filters
$postFilter(results, params) {}, // Function => results, or [Function] => results
body, // The query body
}, {
params, // Default parameters
validateParams, // Object or Function
})
```
### Exposing Named Query
```js
query.expose({
firewall(userId, params) {}, // Function or [Function]
method, // Boolean
publication, // Boolean
unblock, // Boolean
validateParams, // Function or Object
embody // Object which extends the body server-side securely
})
```
### Creating and Exposting Resolvers
```js
// both
const query = createQuery('queryName', () => {});
// server
query.expose({
firewall, // Function or [Function]
});
query.resolve(function (params) {
// this.userId
return [];
});
```
### Using Query
```js
query.setParams({}) // extends current params
```
#### Server-Side
```js
query.clone({params}).fetch();
query.clone({params}).fetchOne();
query.clone({params}).getCount();
```
#### Client-Side
Static:
```js
query.clone({params}).fetch((err, res) => {});
query.clone({params}).fetchOne((err, res) => {});
query.clone({params}).getCount((err, res) => {});
```
Reactive:
```js
const query = userListQuery.clone({params});
const handle = query.subscribe(); // handle.ready()
const data = query.fetch();
const oneData = query.fetchOne();
const handleCount = query.subscribeCount();
const count = query.getCount();
```
#### Caching Named Queries
```js
import {MemoryResultCacher} from 'meteor/cultofcoders:grapher';
// server-side
query.cacheResults(new MemoryResultCacher({
ttl: 60 * 1000, // 60 seconds
}))
```
#### Creating Global Query
```js
Collection.createQuery({
$options, // Mongo Options {sort, limit, skip}
$filters, // Mongo Filters
$filter({filters, options, params}) {}, // Function or [Function]
$postOptions, // {limit, sort, skip}
$postFilters, // any sift() available filters
$postFilter, // Function => results, or [Function] => results
body, // the rest of the object
})
```
#### Exposing Global Query
```js
Collection.expose({
firewall(filters, options, userId) {}, // Function or [Function]
publication, // Boolean
method, // Boolean
blocking, // Boolean
maxLimit, // Number
maxDepth, // Number
restrictedFields, // [String]
restrictLinks, // [String] or Function,
body, // Object or Function(userId) => Object
});
```

139
docs/caching_results.md Normal file
View file

@ -0,0 +1,139 @@
# Caching Results
Grapher is already performant enough to remove this necessity, but what if we want
to squeeze even more performance? What can we do? We cache'em up.
Caching results only works for Named Queries.
Let's say we have some very heavy queries, that do complex sorting
and complex filtering:
```js
export default Meteor.users.createQuery('myFriendsEmails', {
$filter({filters, params}) {
filters.friendIds = params.userId;
},
$options: {
sort: {createdAt: -1}
},
email: 1,
})
```
Ok maybe it's not that complex or heavy, but use your imagination, imagine we have 100,000,000 users
inside the database. And you just want to look for your friends in this big world.
```js
// server-side
import myFriendsQuery from '...';
import {MemoryResultCacher} from 'meteor/cultofcoders:grapher';
myFriendsQuery.expose({
firewall(userId, params) {
params.userId = userId;
}
});
const cacher = new MemoryResultCacher({
ttl: 60 * 1000, // 1 minute caching
});
myFriendsQuery.cacheResults(cacher);
```
That's it. If there is no cache, it fetches the database and stores the result in memory, and after 60s it clears the cache.
Any other calls done in between this time, will hit the cache.
The caching is done by serializing the params and prefixed by query name, in our case, the cache id looks like:
```
myFriendsEmails{userId:"XXX"}
count::myFriendsEmails{userId:"XXX"}
```
The cacher will be used regardless if you are on server or on the client.
The cacher also works with counts. When you use `query.getCount()`
## Implementing your own
The cacher that you provide exposes:
```
fetch(cacheId, {
query: {fetch()},
countCursor: {count()}
})
generateQueryId(queryName, params)
```
If you want to roll-out your own cache that stores the data outside memory, and you want to share in your cloud,
you may want to use `redis` for that.
```js
import {BaseResultCacher} from 'meteor/cultofcoders:grapher';
/**
* Redis Cacher
*/
export default class RedisCacher extends BaseResultCacher {
// this is the default one in case you need to override it
generateQueryId(queryName, params) {
return `${queryName}::${EJSON.stringify(params)}`;
}
// in case of a count cursor cacheId gets prefixed with 'count::'
fetch(cacheId, fetchables) {
const {client, ttl} = this.config;
const cacheData = client.get(cacheId);
if (cacheData) {
return EJSON.parse(cacheData);
}
const data = BaseResultCacher.fetchData(fetchables);
client.set(cacheId, data, 'PX', ttl);
return data;
}
}
```
And use it:
```js
import RedisCacher from 'somewhere';
const cacher = new RedisCacher({
client: redisClient,
ttl: 60 * 1000
});
myFriendsQuery.cacheResults(cacher);
```
## Caching Resolver Queries
As you may have guessed, this works with resolver queries as well, in their case, instead of the actual query,
we pass as `query` parameter an interface containing `fetch()`.
And your cacher, will never receive `cursorCount` inside the `fetchables` object.
Therefore you can use the same paradigms for your cache for resolver queries as well.
## Invalidation
Unfortunately, if you want to invalidate your cache you can do it yourself manually, as this is not implemented,
but since you can hook in your own cacher, you can do whatever you want.
## Conclusion
If you have very heavy and frequent requests to the database, or any type of requests (resolver queries) you can think
about using a cache.

148
docs/denormalization.md Normal file
View file

@ -0,0 +1,148 @@
# Denormalization
Grapher allows you to denormalize your links, leveraging the power of `herteby:denormalize` package.
You can learn more about its capabilities here: https://github.com/Herteby/denormalize
You may need denormalization in such cases where:
1. You want to avoid another database fetch
2. You need ability to perform complex filtering
This has some caveats that we'll explore, let's check out a simple example first:
```js
// Assuming Images of schema: {_id, path, smallThumb, createdAt}
import Images from '...';
Meteor.users.addLinks({
avatar: {
collection: Images,
field: 'avatarId',
type: 'one',
denormalize: {
field: 'avatarCache', // in which field to store the cache
// what fields to cache from Images, this only works for fields and not links
body: {
path: 1,
smallThumb: 1,
}
}
}
})
```
If you did not add this since the beginning of your app, you need to add this into your migration script:
```js
import {migrate} from 'meteor/herteby:denormalize'
migrate('users', 'avatarCache');
```
Now if you are doing the following query:
```js
const user = Meteor.users.createQuery({
avatar: {
smallThumbPath: 1,
}
}).fetchOne()
```
Grapher will check to see if avatar's body is a subbody of the denormalized body. If yes, it will hit the cache,
leading to a single database request.
And the result will be as you expect it to be:
```
{
_id: 'XXX',
avatar: {
_id: 'YYY',
smallThumbPath: '/path/to/file.small.png'
}
}
```
It works very well with nested fields, so you don't have to worry about that.
However, a query like this:
```js
const user = Meteor.users.createQuery({
avatar: {
smallThumbPath: 1,
createdAt: 1,
}
}).fetchOne()
```
Will result in a subsequent database request, because `createdAt` is not in the denormalized body. But if you swap it with `path` then it will hit the cache.
When the user sets a new Avatar, or the `Image` object of that Avatar gets updated. The cache gets automatically updated,
so you don't have to worry about anything. It's magical.
Denormalization works with any type of links `one`, `many`, `meta` whether they are `direct` or `inversed`.
We previously tackled the case where we needed `$postFilters` or `$postFilter` to retrieve some special kind of data:
Let's take a case: we want to retrieve only the users that have reviewed a book of a certain type, and inside `books` collection,
we have `reviewedByUserIds`.
```js
Books.addLinks({
'reviewedByUsers': {
type: 'many',
collection: Meteor.users,
field: 'reviewedByUserIds',
}
})
Meteor.users.addLinks({
'bookReviews': {
collection: Books,
inversedBy: 'reviewedByUsers',
denormalize: {
body: {
type: 1,
},
field: 'bookReviewsCache',
}
}
});
```
*Note that this case may not necessarily be the best example, this is just for illustration,
if you have a better example in mind, let us know so we can update this documentation.*
And now, I want to get all the users that have reviewed books of type `Drama`, because I want
to send them an email about a new book.
```js
const dramaticUsers = Meteor.users.createQuery({
$filters: {
'bookReviewsCache.type': 'Drama'
},
email: 1,
}).fetch();
```
Denormalization comes with a price:
1. It adds hooks to the database so it can properly update it, therefore a change somewhere can result
into additional computation.
2. If you are not careful it can lead to very big caches, which is not what you want unless you favor performance over storage.
The package also supports caching fields and caching counts. You can define those caches outside Grapher without a problem, and specify those fields in your query.
## Caution
If you want to use deep filters, it will not work with denormalized cache, you can use `$postFilter()` method for that.
Because if you put `$filters: {}` inside the body of the cache, it will regard it as a foreign field, and it will fetch the linked Collection for it.
A current limitation for denormalized meta links, is that we will no longer be able to store the `$metadata` inside the nested object, because that
would require additional fetching of the link storage,
## Conclusion
Using denormalization can enable you to do wonderful things inside NoSQL, but also be careful because they come with a cost,
that may not be very noticeable in the beginning. But can also dramatically improve performance at the same time.
I suggest that they should be used to cache things that rarely change such as an user's avatar, or when you need to do
powerful and frequent searches, that otherwise would have consumed more resources.

340
docs/global_exposure.md Normal file
View file

@ -0,0 +1,340 @@
## Global Queries
Global queries are not recommended because they are very hard to secure.
But they are very interesting in what they offer and they can prove to be very useful.
You can expose an API that has access to all or certain parts of your database, without
defining a named query for each.
The difference between a `Named Query` and a `Global Query` is that the later
does not have their form defined on the server, the client can query for anything that he wishes
as long as the query is exposed and respects the security options.
A `Global Query` is as almost feature rich as a `Named Query` with the exception of caching.
#### Real life usage
- You have a public database, you just want to expose it
- You have a multi-tenant system, and you want to give full database access to the tenant admin
- Other cases as well
In order to query for a collection from the client-side and fetch it or subscribe to it. You must expose it.
Exposing a collection does the following things:
- Creates a method called: exposure_{collectionName} which accepts a query
- Creates a publication called: exposure_{collectionName} which accepts a query and uses [reywood:publish-composite](https://atmospherejs.com/reywood/publish-composite) to achieve reactive relationships.
- If firewall is specified, it extends *find* method of your collection, allowing an extra parameter:
```
Collection.find(filters, options, userId);
```
If *userId* is undefined, the firewall and constraints will not be applied. If the *userId* is *null*, the firewall will be applied. This is to allows server-side fetching without any restrictions.
#### Exposing a collection to everyone
```js
// server-side
Meteor.users.expose();
```
This means that any user, from the client can do:
```js
createQuery({
users: {
services: 1, // yes, everything becomes exposed
anyLink: {
anySubLink: {
// and it can go on and on and on
}
}
}
})
```
Ok this is very bad. Let's secure it.
```js
Meteor.users.expose({
restrictedFields: ['services'],
restrictLinks: ['anyLink'],
});
```
Phiew, that's better. But is it? You'll have to keep track of all the link restrictions.
#### Exposing a collection to logged in users
```js
// server-side
Collection.expose({
firewall(filters, options, userId) {
if (!userId) {
throw new Meteor.Error('...');
}
}
});
```
#### Exposure Options
```js
Collection.expose({
// it can also be an array of functions
firewall(filters, options, userId) {
filters.userId = userId;
},
// Allow reactive query-ing
publication: true,
// Allow static query-in
method: true,
// Unblock() the method/publication
blocking: false,
// The publication/method will not allow data fetching for more than 100 items.
maxLimit: 100,
// The publication/method will not allow a query with more than 3 levels deep.
maxDepth: 3,
// This will clean up filters, options.sort and options.fields and remove those fields from there.
// It even removes it from deep filters with $or, $nin, etc
restrictedFields: ['services', 'secretField'],
// Array of strings or a function that has userId
restrictLinks: ['link1', 'link2']
});
```
#### Exposure firewalls are linked
When querying for a data-graph like:
```
{
users: {
comments: {}
}
}
```
It is not necessary to have an exposure for *comments*, however if you do have it, and it has a firewall. The firewall rules will be applied.
The reason for this is security.
Don't worry about performance. We went great lengths to retrieve data in as few MongoDB requests as possible, in the scenario above,
if you do have a firewall for users and comments, both will be called only once, because we only make 2 MongoDB requests.
#### Setting Default Configuration
```
import { Exposure } from 'meteor/cultofcoders:grapher';
// Make sure you do this before exposing any collections.
Exposure.setConfig({
firewall,
method,
publication,
blocking,
maxLimit,
maxDepth,
restrictedFields
});
```
When you expose a collection, it will extend the global exposure methods.
The reason for this is you may want a global limit of 100, or you may want a maximum graph depth of 5 for all your exposed collections,
without having to specify this for each.
Important: if global exposure has a firewall and the collection exposure has a firewall defined as well,
the collection exposure firewall will be applied.
##### Taming The Firewall
```js
// Apply filters based on userId
Collection.expose({
firewall(filters, options, userId) {
if (!isAdmin(userId)) {
filters.isVisible = true;
}
}
});
```
```js
// Make certain fields invisible for certain users
import { Exposure } from 'meteor/cultofcoders:grapher'
Collection.expose({
firewall(filters, options, userId) {
if (!isAdmin(userId)) {
Exposure.restrictFields(filters, options, ['privateData']);
// it will remove all specified fields from filters, options.sort, options.fields
// this way you will not expose unwanted data.
}
}
});
```
#### Restrict certain links by userId
Compute restricted links when fetching the query:
```js
Collection.expose({
restrictLinks(userId) {
return ['privateLink', 'anotherPrivateLink']
}
});
```
## Exposure Body
If *body* is specified, it is first applied on the requested body and then the subsequent rules such as *restrictedFields*, *restrictLinks*
will apply still.
This is for advanced usage and it completes the security of exposure.
By using body, Grapher automatically assumes you have control over what you give,
meaning all firewalls from other exposures for linked elements in this body will be bypassed.
The firewall of the current exposure still executes ofcourse.
#### Basic Usage
```js
Meteor.users.expose({
body: {
firstName: 1,
groups: {
name: 1
}
}
})
```
If you query from the *client-side* something like:
```js
createQuery({
users: {
firstName: 1,
lastName: 1,
groups: {
name: 1,
createdAt: 1,
}
}
})
```
The intersected body will look like:
```
{
firstName: 1,
groups: {
name: 1,
}
}
```
Ok, but what if I want to have a different body based on the userId?
Body can also be a function that takes in an `userId`, and returns an actual body, an `Object`.
```js
Collection.expose({
body(userId) {
let body = { firstName: 1 };
if (isAdmin(userId)) {
_.extend(body, { lastName: 1 })
}
return body;
}
})
```
Deep nesting with other links not be allowed unless your *body* specifies it.
The special fields `$filters` and `$options` are allowed at any link level (including root). However, they will go through a phase of cleaning,
meaning it will only allow you to `filter` and `sort` for fields that exist in the body.
This check goes deeply to verify "$and", "$or", "$nin" and "$not" special MongoDB selectors. This way you are sure you do not expose data you don't want to.
Because, given enough requests, a hacker playing with `$filters` and `$sort` options can figure out a field that you may not want to give him access to.
If the *body* contains functions they will be computed before intersection. Each function will receive userId.
```js
{
linkName(userId) { return {test: 1} }
}
// transforms into
{
linkName: {
test: 1
}
}
```
You can return *undefined* or *false* in your function if you want to disable the field/link for intersection.
```js
{
linkName(userId) {
if (isAdmin(userId)) {
return object;
}
}
}
```
#### Linking Grapher Exposure Bodies
Now things start to get crazy around here!
You can link bodies in your own way and also reference other bodies'links.
Functions are computed on-demand, meaning you can have self-referencing body functions:
```js
// Comments ONE link to Users as 'user'
// Users INVERSED 'user' from Comments AS 'comments'
const commentBody = function(userId) {
return {
user: userBody,
text: 1
}
}
const userBody = function(userId) {
if (isAdmin(userId)) {
return {
comments: commentBody
};
}
return somethingElse;
}
Users.expose({
body: userBody
})
Comments.expose({
body: commentBody
})
```
This will allow requests like:
```js
{
users: {
comments: {
user: {
// It doesn't make much sense for this case
// but you can :)
}
}
}
}
```
## Conclusion
The global queries are a very powerful tool to expose your full database, but unlike `Named Queries` they do
not benefit of `caching`.

77
docs/hypernova.md Normal file
View file

@ -0,0 +1,77 @@
## Hypernova
This is the crown jewl of Grapher. It was named like this because it felt like an explosion of data.
Grapher is very performant. To understand what we're talking about let's take this example of a query
```js
createQuery({
posts: {
categories: {
name: 1,
},
author: {
name: 1,
},
comments: {
$options: {limit: 10},
author: {
name: 1,
}
}
}
})
```
### Queries Counting
In a normal scenario, to retrieve this data graph we need to:
1. Fetch the posts
2. posts.length * Fetch categories for each post
3. posts.length * Fetch author for each post
4. posts.length * Fetch comments for each post
5. posts.length * 10 * Fetch author for each comment
Assuming we have:
- 10 posts
- 2 categories per post
- 1 author per post
- 10 comments per post
- 1 author per comment
We would have blasted the database with:
- Posts: 1
- Categories: 10
- Post authors: 10
- Post comments: 10
- Post comments authors: 100
This means 131 database requests.
Ok, you can cache some stuff, maybe some authors collide, but in order to write a performant code,
you would have to write a bunch of non-reusable code.
But this is just a simple query, imagine something deeper nested. Grapher simply destroys any other
MongoDB "relational" ORM.
### Hypernova to the rescue
How many requests does the Hypernova?
- 1 for Posts
- 1 for all authors inside Posts
- 1 for all categories inside Posts
- 1 for all comments inside Posts
- 1 for all authors inside all comments
The number of database is predictable, because it represents the number of collection nodes inside the graph.
It does this by aggregating filters and then it reassembles data locally.
Not only it makes 5 requests instead of 131, but it smartly re-uses categories and authors at each collection node,
meaning you will have less bandwidth consumed.
Now you understand why this is a revolution for MongoDB.
Keep in mind that Hypernova is only used for static queries. For reactive queries, we still rely on the recursive fetching.

220
docs/introduction.md Normal file
View file

@ -0,0 +1,220 @@
# Welcome
## Installation
```
meteor add cultofcoders:grapher
```
## The 3 Modules
Grapher is composed of 3 main modules, that work together:
### Link Manager
This module allows you to configure relationships between collections and allows you to create denormalized links and resolver links.
### Query
The query module is used for fetching your data in a friendly manner, such as:
```js
createQuery({
users: {
firstName: 1
}
})
```
It abstracts your query into a graph composed of Collection Nodes, Field Nodes and Resolver Nodes,
it uses the **Link Manager** to construct this graph and if the fetching is done server-side (non-reactive queries),
it uses the **Hypernova Module** the crown jewl of Grapher, which heavily minimizes requests to database.
### Exposure
The exposure represents the layer between your queries and the client, allowing you to securely expose your queries,
only to users that have access.
### Your first query
You can use Grapher, without defining any links, for example, let's say you have a method which returns a list of posts.
```js
Meteor.methods({
posts() {
return Posts.find({}, {
fields: {
title: 1,
createdAt: 1,
createdBy: 1,
}
}).fetch();
}
})
```
Transforming this into a Grapher query would simply look like this:
```js
Meteor.methods({
posts() {
const query = Posts.createQuery({
title: 1,
createdAt: 1,
createdBy: 1,
});
return query.fetch();
}
})
```
One of the advantages that Grapher has, is the fact that it forces you to specify the fields you need,
you may find this cumbersome in the beginning, but as your application grows, new fields are added,
fields that need to be protected, you'll find yourself refactoring parts of your code-base which exposed
all the fields.
If, for example, you want to filter or sort your query, we have some special query variables to do so:
```js
Meteor.methods({
posts() {
// Previously Posts.find({isApproved: true}, {sort: '...', fields: '...'});
const query = Posts.createQuery({
$filters: {
isApproved: true,
},
$options: {
sort: {createdAt: -1}
},
title: 1,
createdAt: 1,
createdBy: 1,
});
return query.fetch();
}
})
```
If for example you are searching an element by `_id`, you may have `$filters: {_id: 'XXX'}`, then instead of `fetch()` you
can call `.fetchOne()` so it will return the first element found.
As you may have noticed, the $filters and $options are the ones you pass to `find()`.
The nature of a Query is to be re-usable. For this we introduce a special type of field called `$filter`.
And we allow the query to receive parameters before it executes:
```js
export default Posts.createQuery({
$filter({filters, options, params}) {
filters.isApproved = params.isApproved;
},
$options: {sort: {createdAt: -1}},
title: 1,
createdAt: 1,
createdBy: 1,
});
```
The `$filter()` function receives a single object that contains 3 objects: `filters`, `options`, `params`.
The `filters` and `options` are initially what you provided in `$filters` and `$options` query, they will be empty
if they haven't been specified.
The job of `$filter()` is to extend/modify `filters` and `options`, based on params.
Lets see how we can use that query:
```js
// assuming you exported it from '...'
import postListQuery from '...';
Meteor.methods({
posts() {
return postListQuery.clone({
isApproved: true
}).fetch()
}
})
```
Whenever we want to use a modular query, we have to `clone()` it so it creates a new standalone instance,
that does not affect the exported one. The `clone()` accepts `params` as argument.
Those `params` will be passed to the `$filter` function.
You could also use `setParams()` to configure parameters:
```js
import postListQuery from '...';
Meteor.methods({
posts() {
const query = postListQuery.clone();
query.setParams({
isApproved: true,
});
return query.fetch();
}
})
```
A query can be smart enough to know what parameters it needs, for this we can use the awesome `check` library from Meteor:
http://docs.meteor.com/api/check.html
```js
import {Match} from 'meteor/check';
export default Posts.createQuery({
$filter({filters, options, params}) {
filters.isApproved = params.isApproved;
if (params.authorId) {
filters.authorId = params.authorId;
}
},
...
}, {
validateParams: {
isApproved: Boolean,
authorId: Match.Maybe(String),
}
});
```
But you can craft your own validation:
```js
{
validateParams(params) {
if (somethingIsWrong) {
throw new Meteor.Error('invalid-params', 'Explain why');
}
}
}
```
Note: params validation is done prior to fetching the query.
And if you want to set some default parameters:
```js
export default Posts.createQuery({...}, {
params: {
isApproved: true,
}
});
```
## Conclusion
This is the end of our introduction. As we can see, we can make queries modular and this already gives us
a big benefit. By abstracting them into their own modules we can keep our methods neat and clean,
and we haven't even arrived to the good parts.

170
docs/linker_engine.md
View file

@ -0,0 +1,170 @@
# Linker Engine
The linker engine is composed of 3 components:
- Definition of Links
- Linked Data Retrieval
- Setting Links
Let's explore how we can play with `Linked Data Retrieval` and `Setting Links`, assumming we already defined our links
using `addLinks()` method.
Each collection gets extended with a `getLink` function:
```js
const linker = Collection.getLink(collectionItemId, 'linkName');
```
A real life example (assuming you have a `direct` or `inversed` link with groups)
```js
const userGroupLinker = Meteor.users.getLink(userId, 'groups');
```
Linker is polymorphic and it allows you to do almost magical things. It is so flexible that it will
allow you to use it very naturally.
## Linked Data Retrieval
```js
const userGroupLinker = Meteor.users.getLink(userId, 'groups');
// fetching the groups for that user:
const groups = userGroupLinker.find().fetch();
// or for ease of use
const groups = userGroupLinker.fetch();
// or to filter the nested elements
const groups = userGroupLinker.find(filters, options).fetch();
// and again simpler
const groups = userGroupLinker.fetch(filters, options);
// and if you want to performantly get a count()
// because find() returns a good ol' Mongo.Cursor
const groups = userGroupLinker.find(filters, options).count();
```
This works with any kind of links from any side.
## Setting Links
This allows you to very easily link collections to each other, without relying on knowing the fields and how are they stored.
It also allows you to set links from any place `direct` and `inversed`, of any type `one` or `many` and `meta` links as well:
#### "One" Relationships
Performing a `set()` will automatically execute the update or insert in the database.
```js
const userPaymentProfileLink = Meteor.users.getLink(userId, 'paymentProfile');
userPaymentProfileLink.set(paymentProfileId);
// but it also works if you have the object directly if it has _id, for ease of use:
userPaymentProfileLink.set(paymentProfile);
// it works from the other side as well
const paymentProfileUserLink = PaymentProfiles.getLink(paymentProfileId, 'user');
paymentProfileUserLink.set(userId); // or a user object that contains `_id`
```
You can also `set` objects that aren't in the database yet:
```js
const userPaymentProfileLink = Meteor.users.getLink(userId, 'paymentProfile');
userPaymentProfileLink.set({
last4digits: '1234',
});
```
This will insert into the `PaymentProfiles` collection and link it to user and it works from both `direct` and `inversed` side as well.
To remove a link for a `one` relationship (no arguments required):
```js
userPaymentProfileLink.unset();
// or
paymentProfileUserLink.unset();
```
#### "Many" Relationships
Same principles as above apply, with some minor changes, this time we use `add` and `remove`
```js
const userGroupsLink = Meteor.users.getLink(userId, 'groups');
userGroupsLink.add(groupId);
userGroupsLink.add(group); // object containing an _id
userGroupsLink.add({
name: 1,
}); // will add the group to the database and link it accordingly
```
The methods `add()` and `remove()` also accept arrays
```js
userGroupsLink.add([
groupId1,
groupId2
]);
userGroupsLink.remove(groupIds)
```
The same logic applies, you can add array of objects that contain `_id` or array of objects without `_id`, or a mixture of them.
Ofcourse, the `remove()` cannot accept objects without `_id` as it makes no sense to do so.
#### "Meta" Relationships
Now things get very interesting, because `metadata` allows us to store additional information about the link,
it lets us **describe** the relationship. And again, this works from `direct` and `inversed` side as well, with the
same principles described as above.
The `add()` and `set()` allow an additional parameter `metadata`:
```js
// one
const userPaymentProfileLink = Meteor.users.getLink(userId, 'paymentProfile');
userPaymentProfileLink.set(paymentProfileId, {
createdAt: new Date()
});
// many
const userGroupsLink = Meteor.users.getLink(userId, 'groups');
userGroupsLink.add(groupId, {
createdAt: new Date(),
})
// if you add multiple objects, they will receive the same metadata
userGroupsLink.add([groupId1, groupId2], {
createdAt: new Date(),
})
```
Updating existing metadata:
```js
// one
const userPaymentProfileLink = Meteor.users.getLink(userId, 'paymentProfile');
userPaymentProfileLink.metadata({
updatedAt: new Date()
});
// many
userGroupsLink.metadata(groupId, {
createdAt: new Date(),
})
userGroupsLink.metadata([groupId1, groupId2], {
createdAt: new Date(),
})
```
Updating metadata only works with strings or objects that contain `_id`, and it works from both sides.
## Conclusion
By using this Programatic API to set your links instead of relying on updates, it makes your code much simpler to read,
much easier to migrate in the future to a new database relational model.

453
docs/linking_collections.md
View file

@ -0,0 +1,453 @@
# Linking Collections
Let's learn what types of links we can do between collections, and what is the best way to do them.
First, we begin with an illustration of the power of Grapher:
Let's assume our posts, contain a field, called `authorId` which represents an actual `_id` from `Meteor.users`,
if you wanted to get the post and the author's name, you had to first fetch the post,
and then get the author's name based on his `_id`.
Something like:
```js
Meteor.methods({
getPost({postId}) {
let post = Posts.findOne(postId, {
fields: {
title: 1,
createdAt: 1,
authorId: 1,
}
});
if (!post) { throw new Meteor.Error('not-found') }
const author = Meteor.users.findOne(post.authorId, {
fields: {
firstName: 1,
lastName: 1
}
});
Object.assign(post, {author});
return post;
}
})
```
With Grapher, your code above is transformed to:
```js
Meteor.methods({
getPost({postId}) {
let post = Posts.createQuery({
$filters: {_id: postId},
title: 1,
createdAt: 1,
author: {
firstName: 1,
lastName: 1
}
});
return post.fetchOne();
}
})
```
This is just a simple illustration, imagine the scenario, in which you had comments,
and the comments had authors, and you needed their avatar. Your code can easily
grow to many lines of code, and it will be much less performant because of the many round-trips you do to
the database.
## Linking Types
To make the link illustrated in the example above, we create a separate `links.js` file that is
imported separately outside the collection module. We'll understand later why.
```js
// file: /imports/db/posts/links.js
import Posts from '...';
Posts.addLinks({
'author': {
type: 'one',
collection: Meteor.users,
field: 'authorId',
}
})
```
That was it. You created the link, and now you can use the query illustrated above.
We decided to choose `author` as a name for our link and `authorId` the field to store it in, but any string will do.
### Inversed Links
Because we linked `Posts` with `Meteor.users` it means that we can also get `posts` if we are in a User.
But because the link is stored in `Posts` we need a new type of linking, and we call it `Inversed Link`
```js
// file: /imports/db/users/links.js
import Posts from '...';
Meteor.users.addLinks({
'posts': {
collection: Posts,
inversedBy: 'author'
}
})
```
`author` represents the link name that was defined inside Posts. By defining inversed links we can do:
```js
Meteor.users.createQuery({
posts: {
title: 1
}
})
```
### One and Many
Above you've noticed a `type: 'one'` in the link definition, but let's say we have a `Post` that belongs to many `Categories`,
which have their own collection into the database. This means that we need to relate with more than a single element.
```js
// file: /imports/db/posts/links.js
import Posts from '...';
import Categories from '...';
Posts.addLinks({
'author': { ... },
'categories': {
type: 'many',
collection: Categories,
field: 'categoryIds',
}
})
```
In this case, `categoryIds` is an array of Strings, each String, representing `_id` from `Categories` collection.
And, ofcourse, you can also create an inversed link from `Categories`, so you can use it inside the `query`
```js
// file: /imports/db/posts/links.js
import Categories from '...';
import Posts from '...';
Categories.addLinks({
'posts': {
collection: Posts,
inversedBy: 'categories'
}
})
```
## Meta Links
We use a `meta` link when we want to add additional data about the relationship. For example,
a user can belong in a single `Group`, but we need to know when he joined that group and what roles he has in it.
```js
// file: /imports/db/users/links.js
import Groups from '...'
Meteor.users.addLinks({
group: {
type: 'one',
collection: Groups,
field: 'groupLink',
metadata: true,
}
})
```
Notice the new option `metadata: true` this means that `groupLink` is no longer a `String`, but an `Object` that looks like this:
```
// inside a Meteor.users document
{
...
groupLink: {
_id: 'XXX',
roles: 'ADMIN',
createdAt: Date,
}
}
```
Let's see how this works out in our query:
```js
const user = Meteor.users.createQuery({
$filters: {_id: userId},
group: {
name: 1,
}
}).fetchOne()
```
`user` will look like this:
```
{
_id: userId,
group: {
$metadata: {
roles: 'ADMIN',
createdAt: Date
},
name: 'My Funky Group'
}
}
```
We store the metadata of the link inside a special `$metadata` field. And this works from inversed side as well:
```js
Groups.addLinks({
users: {
collection: Meteor.users,
inversedBy: 'group'
}
});
const group = Groups.createQuery({
$filters: {_id: groupId},
name: 1,
users: {
firstName: 1,
}
}).fetchOne()
```
`group` will look like:
```
{
_id: groupId,
name: 'My Funky Group',
users: [
{
$metadata: {
roles: 'ADMIN',
createdAt: Date
},
_id: userId,
firstName: 'My Funky FirstName',
}
]
}
```
The same principles apply to `meta` links that are `type: 'many'`, if we change that in the example above.
The storage field will look like:
```
{
groupLinks: [
{_id: 'groupId', roles: 'ADMIN', createdAt: Date},
...
]
}
```
And they work the same as you expect, from the inversed side as well.
I know what question comes to your mind right now, what if I want to put a field inside the metadata,
I want to store who added that user (`addedBy`) to that link and fetch it, how do we do ?
Currently, this is not possible, because it has deep implications with how the Hypernova Module works,
but you can achieve this in a very performant way, by abstracting it into a separate collection:
```js
// file: /imports/db/groupUserLinks/links.js
import Groups from '...';
import GroupUserLinks from '...';
GroupUserLinks.addLinks({
user: {
type: 'one',
collection: Meteor.users,
field: 'userId'
},
adder: {
type: 'one',
collection: Meteor.users,
field: 'addedBy'
},
group: {
type: 'one',
collection: Meteor.users,
field: 'groupId'
}
})
// file: /imports/db/users/links.js
Meteor.users.addLinks({
groupLink: {
collection: GroupUserLinks,
type: 'one',
field: 'groupLinkId',
}
})
```
And the query will look like this:
```js
Meteor.users.createQuery({
groupLink: {
group: {
name: 1,
},
adder: {
firstName: 1
},
roles: 1,
createdAt: 1,
}
})
```
## Link Loopback
No one stops you from linking a collection to itself, say you have a list of friends which are also users:
```js
Meteor.users.addLinks({
friends: {
collection: Meteor.users,
type: 'many',
field: 'friendIds',
}
});
```
Say you want to get your friends, and friends of friends, and friends of friends of friends!
```js
Meteor.users.createQuery({
$filters: {_id: userId},
friends: {
nickname: 1,
friends: {
nickname: 1,
friends: {
nickname: 1,
}
}
}
});
```
## Uniqueness
The `type: 'one'` doesn't necessarily guarantee uniqueness from the inversed side.
For example, if we have inside `Comments` a link with `Posts` with `type: 'one'` inside postId,
it does not mean that when I fetch the posts with comments, comments will be an array.
But if you want to have a one to one relationship, and you want grapher to give you an object,
instead of an array you can do so:
```js
Meteor.users.addLinks({
paymentProfile: {
collection: PaymentProfiles,
inversedBy: 'user'
}
});
PaymentProfiles.addLinks({
user: {
field: 'userId',
collection: Meteor.users,
type: 'one',
unique: true
}
})
```
Now fetching:
```js
Meteor.users.createQuery({
paymentProfile: {
type: 1,
last4digits: 1,
}
});
```
`paymentProfile` inside `user` will be an object because it knows it should be unique.
## Data Consistency
This is referring to having consistency amongst links.
Let's say I have a `thread` with multiple `members` from `Meteor.users`. If a `user` is deleted from the database, we don't want to keep unexisting references.
So after we delete a user, all threads containing that users should be cleaned.
This is done automatically by Grapher so you don't have to deal with it.
The only rule is that `Meteor.users` collection needs to have an inversed link to `Threads`.
In conclusion, if you want to benefit of this, you have to define inversed links for every direct links.
## Autoremoval
```js
Meteor.users.addLinks({
'posts': {
collection: Posts,
inversedBy: 'author',
autoremove: true
}
});
```
After you deleted a user, all the links that have `autoremove: true` will be deleted.
This works from the `direct` side as well, not only from `inversed` side.
## Indexing
As a rule of thumb, you must index all of your links. Because that's how you achieve absolute performance.
This is not done by default, to allow the developer flexibility, but you can do it simply enough from the direct side definition of the link:
```js
PaymentProfiles.addLinks({
user: {
field: 'userId',
collection: Meteor.users,
type: 'one',
unique: true,
index: true,
}
})
```
The index is applied only on the `_id`, meaning that if you have `meta` links, other fields present in that object will not be indexed.
If you have `unique: true` set, the index will also apply a unique constraint to it.
## Top Level Fields
Grapher currently supports only top level fields for storing data. One of the reasons it doesn't allow nested fields
is to enforce the developer to think relational and eliminate large and complex documents by abstracting them into collections.
In the future, this limitation may change, but for now you can work around this and keep your code elegant.
## Conclusion
Using these simple techniques, you can create a beautiful database schemas inside MongoDB that are relational and very simple to fetch,
you will eliminate almost all your boilerplate code around this and allows you to focus on more important things.
On top of a cleaner code you benefit from the `Hypernova Module` which minimizes the database requests to a
predictable number (no. of collection nodes).

398
docs/named_queries.md
View file

@ -0,0 +1,398 @@
# Named Queries
Before we explain what they are, we need to understand an alternative way of creating the queries.
## Alternative Creation
Currently, we only saw how to create a query starting from a collection, like:
```js
Meteor.users.createQuery(body, options);
```
But Grapher also exposes a `createQuery` functionality:
```js
import {createQuery} from 'meteor/cultofcoders:grapher';
createQuery({
users: {
profile: 1,
}
})
```
The first key inside the object needs to represent an existing collection name:
```js
const Posts = new Mongo.Collection('posts');
// then we can do:
createQuery({
posts: {
title: 1
}
})
```
## What are Named Queries ?
As the name implies, they are a query that are identified by a `name` (No shit Sherlock).
The difference is that they accept a `string` as the first argument.
The `Named Query` has the same API as a normal `Query`, we'll understand the difference in this documentation.
```js
// file: /imports/api/users/queries/userAdminList.js
export default Meteor.users.createQuery('userAdminList', {
$filters: {
roles: {$in: 'ADMIN'}
},
name: 1,
})
```
If you would like to use this query you have two ways:
```js
import userAdminListQuery from '/imports/db/users/queries/userAdminList.js';
const admins = userAdminListQuery.fetch();
```
Or you could use `createQuery`:
```js
import {createQuery} from 'meteor/cultofcoders:grapher';
const admins = createQuery({
userAdminList: {},
}).fetch();
// will return a clone of the named query if it is already loaded
```
Now notice the {}. Because `Named Queries` have their form well defined, they don't allow you
to specify fields directly, however they allow you to specify parameters when using `createQuery`
Because the default `$filter()` allows as params `filters` and `options` for the top collection node,
you can do something like:
```js
const admins = createQuery({
userAdminList: {
options: {createdAt: -1}
},
}).fetch();
```
## Always go modular
Ofcourse the recommended approach is to always use `modular`. Creating queries in files, and importing them accordingly.
The reason we expose such functionality is because if you want to use Grapher as an HTTP API, you do not have access
to the collections, so this becomes very handy, as you can transform a JSON request into a query.
Therefore, in such a case, a modular approach will not work.
In the client-side of Meteor it is recommended to always extract your queries into their own module, and import and clone them for use.
## But why ?
Why do we need another string? What is the purpose?
It's because we are slowly beginning to cross to the client-side domain, which
is a dangerous and nasty place and we need to expose our queries in a very cautious and secure manner.
## Special $body
`Named Queries` allow a special `$body` parameter. Let's see an advanced query:
```js
const fullPostList = Posts.createQuery('fullPostList', {
title: 1,
author: {
firstName: 1,
lastName: 1,
},
comments: {
text: 1,
createdAt: 1,
author: {
firstName: 1,
lastName: 1,
}
}
})
```
There will be situations where you initially want to show less data for this, maybe just the `title`, but if the user,
clicks on that `title` let's say we want it to expand and then we need to fetch additional data.
The way to do it is to use `$body` which intersects with the allowed data graph:
```js
fullPostsList.clone({
$body: {
title: 1,
author: {
firstName: 1,
services: 1, // will be removed after intersection, and not queried
},
otherLink: {} // will be removed after intersection, and not queried
}
})
```
This will only fetch the intersection, the transformed body will look like:
```js
{
title: 1,
author: {firstName: 1}
}
```
When we learn about exposure you will fully understand why the `$body` special parameter is useful.
Be careful, if you use validation for params (and you should) to also add `$body` inside it:
```js
import {Match} from 'meteor/check';
const fullPostList = Posts.createQuery('fullPostList', {}, {
validateParams: {
$body: Match.Maybe(Object),
}
});
```
## Exposure
We are now crossing the bridge to client-side.
We want our application users to be able of using these queries we define stand-alone. We initially said that Grapher
will become the `Data Fetching Layer` inside Meteor. Therefore, we will no longer rely on methods to give us the data from
our queries, even if it's still possible and up to you to decide, but this gives you many advantages, such as caching and performance monitoring.
As you may have guessed, the exposure needs to happen on the server, but the query can be imported on the client.
Let's say we want only Admins to be able to query our `userAdminList` query:
```js
// file: /imports/api/users/queries/userAdminList.js
export default Meteor.users.createQuery('userAdminList', {
$filters: {
roles: {$in: 'ADMIN'}
},
name: 1,
})
```
Alternatively, you could have done:
```js
// file: /imports/api/users/queries/userAdminList.js
export default createQuery('userAdminList', {
users: {
$filters: {
roles: {$in: 'ADMIN'}
},
name: 1,
}
})
```
```js
// server-side (make sure it's imported from somewhere on the server only)
// file: /imports/api/users/queries/userAdminList.expose.js
import userAdminListQuery from './userAdminList';
userAdminListQuery.expose({
firewall(userId, params) {
if (!Roles.userIsInRole(userId, 'ADMIN')) {
throw new Meteor.Error('not-allowed');
}
// in the firewall you also have the ability to modify the parameters
// that are going to hit the $filter() function in the query
}
})
```
## Client-side
Let's use it on the client side:
```js
// client side
import userAdminListQuery from '/imports/api/users/queries/userAdminList.js';
userAdminListQuery.clone().fetch((err, res) => {
// do something with err, res
// it will be an error if the current user is not an admin
});
```
A beautiful part of Grapher lies in it's polymorphic ability, you can have a reactive query, just by using the same API:
```js
import {Tracker} from 'meteor/tracker';
import userAdminListQuery from '/imports/api/users/queries/userAdminList.js';
const query = userAdminListQuery.clone();
const subscriptionHandle = query.subscribe();
// if we did subscribe, we no longer need to supply a callback for fetch()
// as the query morphed into a reactive query
Tracker.autorun(() => {
if (subscriptionHandle.ready()) {
console.log(query.fetch());
query.unsubscribe();
}
})
```
You can do `meteor add tracker` if it complains it's missing `Tracker`.
You can also use `fetchOne()` on the client as well, even supply a callback:
```js
userAdminListQuery.clone().fetchOne((err, user) => {
// do something
});
```
You can fetch static queries using promises:
```js
const users = await userAdminListQuery.clone().fetchSync();
const user = await userAdminListQuery.clone().fetchOneSync();
```
## Counters
You can ofcourse use the same paradigm to use counts:
```js
// static query
query.getCount((err, count) => {
// do something
});
const count = await query.getCountSync();
// reactive counts
const handle = query.subscribeCount();
Tracker.autorun(() => {
if (handle.ready()) {
console.log(query.getCount());
query.unsubscribeCount();
}
});
```
We do not subscribe to counts by default because they may be too expensive, but if you need them,
feel free to use them.
## Behind the scenes
When we are dealing with a static query (non-reactive), we make a call to the method that has been created when
we did `query.expose()`, the firewall gets applied and returns the result from the query.
If we have a reactive query, `query.expose()` creates a publication end-point, and inside it, uses the `reywood:publish-composite` package
to automatically create the proper publication.
When we do `query.fetch()` on a reactive query, it gives you a complete data graph with the items in the same form as you
would have received from a static query. When an element inside your data graph changes, the `fetch()` function is re-run
if you are inside a `Tracker.autorun()` or a reactive context, because in the back, we do `find().fetch()` on client-side collections.
## Warning
If you want your client-side **reactive** queries to work as expected you need to make sure that the `addLinks` and `addReducers` are loaded on the client as well.
Because the data assembly is now done on the client.
## Expose Options
```js
query.expose({
// Secure your query
firewall(userId, params) {
// you can modify the parameters here
},
// Allow the query to be fetched statically
method: true, // default
// Allow the query to be fetched reactively (for heavy queries, you may want to set it to false)
publication: true, // default
// Unblocks your method (and if you have .unblock() in publication context it also unblocks it)
unblock: true, // default
// This can be an object or a function(params) that you don't want to expose it on the client via
// The query options as it may hold some secret business data
// If you don't specify it the default validateParams from the query applies, but not both!
// If you allow subbody requests, don't forget to add {$body: Match.Maybe(Boolean)}
validateParams: {},
// This deep extends your graph's body before processing it.
// For example, you want a hidden $filter() functionality, or anything else.
embody: {
$filter({filters, params}) {
// do something
},
aLink: {
$filter() {
// works with links as well, because it's a deep extension
}
}
}
})
```
## Resolvers
Named Queries have the ability to morph themselves into a function that executes on the server.
There are situations where you want to retrieve some data, that is not necessarily inside a collection,
or it needs additional operations to give you the correct result.
For example, let's say you want to provide the user with some analytic results, that perform some heavy
aggregations on the database, or call some external APIs.
```js
// shared code between client and server
const getAnalytics = createQuery('getAnalytics', () => {}, {
validateParams: {} // Object or Function
});
```
```js
// server code only
import getAnalyticsQuery from './getAnalytics';
getAnalyticsQuery.expose({
firewall(userId, params) {
// ...
},
validateParams: {} // Object or Function that you don't want exposed
});
getAnalyticsQuery.resolve(function(params) {
// perform your magic here
// you are in `Meteor.method` context, so you have access to `this.userId`
})
```
The advantage of doing things like this, instead of relying on a method, is that you are now
using a uniform layer for fetching results, and on top of that, you can easily cache them.
## Mix'em up
The `firewall` can also be an array of functions. Allowing you to easily expose a query like:
```js
function checkLoggedIn(userId, params) {
if (!userId) {
throw new Meteor.Error('not-allowed');
}
}
query.expose({
firewall: [checkLoggedIn, (userId, params) => {
params.userId = userId;
// other stuff here if you want
}]
})
```
## Conclusion
We can now safely expose our queries to the client, and the client can use it in a simple and uniform way.
By this stage we already understand how powerful Grapher really is, but it still has some tricks.

100
docs/outside_grapher.md Normal file
View file

@ -0,0 +1,100 @@
## Grapher as an API
If you like Grapher, you can use it in any other language/medium.
You can expose it as an HTTP API, or a DDP API (as a Meteor.method()) for example,
because in ReactNative you have ways to connect to Meteor with:
https://www.npmjs.com/package/react-native-meteor#meteor-collections
Basically what Grapher needs to properly execute is the query,
and if you have firewalls, you need to manually handle authorization yourself.
### Exposing an HTTP API
```js
// This is how you can create a sample endpoint
import {createQuery} from 'meteor/cultofcoders:grapher';
import Picker from 'meteor/meteorhacks:picker';
import {EJSON} from 'meteor/ejson';
import {Meteor} from 'meteor/meteor';
import bodyParser from 'body-parser';
const grapherRoutes = Picker.filter(function () {
return true;
});
grapherRoutes.middleware(bodyParser.raw({
'type': 'application/ejson',
}));
grapherRoutes.route('/grapher', function (req, res) {
const body = req.body.toString();
const data = EJSON.parse(body);
// lets say this is a named query that looks like
// {getUserList: params}
const {query} = data;
// authorize the user somehow
// it's up to you to extract an userId
// or something else that you use for authorization
const actualQuery = createQuery(query);
// if it's not a named query and the collection is not exposed, don't allow it.
if (actualQuery.isGlobalQuery && !actualQuery.collection.__isExposedForGrapher) {
throw new Meteor.Error('not-allowed');
}
try {
const data = actualQuery.fetch({
// the userId (User Identification) that hits the firewalls
// user id can be anything, an API key maybe, not only a 'string'
// it's up to you and your firewalls to decide
userId: 'XXX'
});
res.statusCode = 200;
res.end(EJSON.stringify({
data,
}));
} catch (e) {
res.statusCode = 500;
res.end(EJSON.stringify({
error: e.reason,
}));
}
})
```
Now you can do HTTP requests of `Content-Type: application/ejson` to http://meteor-server/grapher and retrieve data.
If you want to use Meteor's Methods as an HTTP API to also handle method calls, take a look here:
- https://github.com/cult-of-coders/fusion
And more closely: https://github.com/cult-of-coders/fusion/blob/7ec5cd50c3a471c0bdd65c9fa482124c149dc243/fusion/server/route.js
### Exposing a DDP Method
```js
import {createQuery} from 'meteor/cultofcoders:grapher';
Meteor.methods({
'grapher'(query) {
const actualQuery = createQuery(query);
if (actualQuery.isGlobalQuery && !actualQuery.collection.__isExposedForGrapher) {
throw new Meteor.Error('not-allowed');
}
return actualQuery.fetch({
userId: this.userId,
})
}
})
```
## Conclusion
Nothing stops you from using Grapher outside Meteor!

375
docs/query_options.md Normal file
View file

@ -0,0 +1,375 @@
# Query Options
Let's learn what a query can do, we know it can fetch related links, but it has some
interesting features.
## Nested fields
Most likely you will have documents which organize data inside an object, such as `user` may have a `profile` object
that stores `firstName`, `lastName`, etc
Grapher automatically detects these fields, as long as there is no link named `profile`:
```js
const user = Meteor.users.createQuery({
$filters: {_id: userId},
profile: {
firstName: 1,
lastName: 1,
}
}).fetchOne();
```
Now `user` will look like:
```
{
_id: userId,
profile: {
firstName: 'John',
lastName: 'Smith',
}
}
```
If you wanted to fetch the full `profile`, simply use `profile: 1` inside your query body. Alternatively,
you can also use `'profile.firstName': 1` and `'profile.lastName': 1` but it's less elegant.
## Deep filtering
Lets say we have a `Post` with `comments`:
```js
import {Comments, Posts} from '/imports/db';
Posts.addLinks({
comments: {
collection: Comments,
inversedBy: 'postId',
}
})
Comments.addLinks({
post: {
type: 'one',
field: 'postId',
collection: Posts,
},
});
```
If any bit of the code written above creates confusion, try reading again the `Linking Collections` part of the documentation.
We already know that we can query with `$filters`, `$options`, `$filter` and have some parameters.
The same logic applies for child collection nodes:
```js
Posts.createQuery({
title: 1,
comments: {
$filters: {
isApproved: true,
},
text: 1,
}
})
```
The query above will fetch as `comments` only the ones that have been approved.
The `$filter` function share the same `params` across all collection nodes:
```js
export default Posts.createQuery({
$filter({filters, params}) {
if (params.lastWeekPosts) {
filters.createdAt = {$gt: date}
}
},
$options: {createdAt: -1},
title: 1,
comments: {
$filter({filters, params}) {
if (params.approvedCommentsOnly) {
filters.isApproved = true;
}
},
$options: {createdAt: -1},
text: 1,
}
})
```
```js
const postsWithComments = postListQuery.clone({
lastWeekPosts: true,
approvedCommentsOnly: true
}).fetch();
```
### Default $filter()
The $filter is a function that defaults to:
```js
function $filter({filters, options, params}) {
if (params.filters) {
Object.assign(filters, params.filters);
}
if (params.options) {
Object.assign(filters, params.options)
}
}
```
Which basically means you can easily configure your filters and options through params:
```js
const postsQuery = Posts.createQuery({
title: 1,
});
const posts = postQuery.clone({
filters: {isApproved: true},
options: {
sort: {createdAt: -1},
}
}).fetch();
```
If you like to disable this functionality, add your own $filter() function or use a dummy one:
```js
{
$filter: () => {},
}
```
Note the default $filter() only applies to the top collection node, otherwise we would have headed into a lot of trouble.
### Pagination
There is a special field that extends the pre-fetch filtering process, and it's called `$paginate`:
```js
const postsQuery = Posts.createQuery({
$filter({filters, params}) {
filters.isApproved = params.postsApproved;
},
$paginate: true,
title: 1,
});
const page = 1;
const perPage = 10;
const posts = postsQuery.clone({
postsApproved: true,
limit: perPage,
skip: (page - 1) * perPage
}).fetch()
```
This is mostly used for your convenience, as pagination is a common used technique and makes your code easier to read.
Note that it doesn't override the $filter() function, it just applies `limit` and `skip` to the options, before $filter() runs.
### Meta Filters
Let's say we have `Users` that belong in `Groups` and they have some roles attached in the link description:
```js
import {Groups} from '/imports/db';
Meteor.users.addLinks({
groups: {
type: 'many',
collection: Groups,
field: 'groupLinks',
metadata: true,
}
});
Groups.addLinks({
users: {
collection: Meteor.users,
inversedBy: 'groups',
}
})
```
Let's assume the groupLinks looks like this:
```js
[
{
_id: 'groupId',
roles: ['ADMIN']
}
]
```
And you want to query users and fetch only the groups he is admin in:
```js
const users = Meteor.users.createQuery({
name: 1,
groups: {
$filters: {
$meta: {
roles: {$in: 'ADMIN'}
}
}
}
}).fetch()
```
But what if you want to fetch the groups and all their admins ? Easy.
```js
const groups = Groups.createQuery({
name: 1,
users: {
$filters: {
$meta: {
roles: {$in: 'ADMIN'}
}
}
}
}).fetch()
```
We have gone through great efforts to support such functionality, but it makes our code so easy to read and it doesn't impact performance.
## Post Filtering
This concept allows us to filter/manipulate data after we received it.
For example, what if you want to get the users that are admins in at least one group:
```js
const users = Meteor.users.createQuery({
$postFilters: {
'groups.$metadata.roles': {$in: 'ADMIN'},
},
name: 1,
groups: {
name :1,
}
}).fetch()
```
If you had a `many` relationship without `metadata` your $postFilters would look like:
```
{
'groups.roles': {$in: 'ADMIN'},
}
```
The `$postFilters` option uses the `sift` npm library (https://www.npmjs.com/package/sift) to make your filters look like mongo filters.
In addition to `$postFilters` we've also got `$postOptions` that allows:
- limit
- sort
- skip
They work exactly like you expect from $options, the difference is that they are applied after the data has been fetched.
```js
const users = Meteor.users.createQuery({
$postOptions: {
sort: {'groups.name': 1},
},
name: 1,
groups: {
name :1,
}
}).fetch()
```
And to offer you full flexibility, we also allow a `$postFilter` function that needs
to return the new set of results.
```js
const users = Meteor.users.createQuery({
$postFilter(results, params) {
if (params.mustHaveGroupsAsAdmin) {
return results.filter(r => {
// your filter goes here.
});
}
},
name: 1,
groups: {
name: 1,
}
}, {
params: {
mustHaveGroupsAsAdmin: true
},
}).fetch()
```
Note the fact that they only work for top level nodes, not for child collection nodes unlike `$filters` and `$options`.
This type of queries that rely on post processing, can prove to be costly in some cases, because it will still fetch the users from the database
that don't have a group in which they are admin. There are alternatives to this to this in the `Denormalize` section of the documentation.
It really depends on your context, but `$postFilters`, `$postOptions` and `$postFilter` can be very useful in some cases.
## Counters
If you want just to return the number of top level documents a query has:
```js
query.getCount()
```
This will be very useful for pagination when we reach the client-side domain.
## Mix'em up
The options `$filter` and `$postFilter` also allow you to provide an array of functions:
```js
function userContext({filters, params}) {
if (!params.userId) {
throw new Meteor.Error('not-allowed');
}
filters.userId = params.userId;
}
const posts = Posts.createQuery({
$filter: [userContext, ({filters, options, params}) => {
// do something
}],
$postFilter: [someOtherFunction],
}).fetch()
```
The example above is just to illustrate the possibility, in order to ensure that a `userId` param is sent you will use `validateParams`
```js
Posts.createQuery({
$filter({filters, options, params}) {
filters.userId = params.userId;
},
title: 1,
}, {
validateParams: {
userId: String
}
})
```
Validating params will also protect you from injections such as:
```js
const query = postLists.clone({
userId: {$nin: []},
})
```
When we cross to the client-side domain we need to be very wary of these type of injections.
## Conclusion
Query is a very powerful tool, very flexible, it allows us to do very complex things that would have taken us a lot of time
to do otherwise.

196
docs/reducers.md Normal file
View file

@ -0,0 +1,196 @@
# Reducers
The reducers are a sort of "smart fields" which allow you to compose different results
from your query.
To achieve this we extend `Mongo.Collection` with an `addReducer()` method:
```js
Collection.addReducer({
reducerName: {
body: graphDependencyBody,
reduce(object) {
return value; // can be anything, object, date, string, number, etc
}
}
})
```
## Basics
```js
Meteor.users.addReducers({
fullName: {
body: {
profile: {
firstName: 1,
lastName: 1
}
},
reduce(object) {
const {profile} = object;
return `${profile.firstName} ${profileLastName}`;
}
}
})
```
Query:
```js
const user = Meteor.users.createQuery({
fullName: 1,
}).fetchOne();
```
Results to:
```
{
_id: 'XXX',
fullName: 'John Smith',
}
```
## Reducers and links
Easily grab the data from your links (as deep as you want them), if you want to reduce it.
```js
Meteor.users.addReducers({
groupNames: {
body: {
// assuming you have a link called groups
groups: { name: 1 }
},
reduce(object) { // a pure function that returns the data
return object.groups.map(group => group.name).join(',')
}
}
})
```
Query:
```js
const user = Meteor.users.createQuery({
groupNames: 1,
}).fetchOne();
```
Result:
```
{
_id: 'XXX',
groupNames: ['Group 1', 'Group 2'],
}
```
Note that `groups: []` is not present in your result set. This is because we detect the fact that you
did not include it in the body of your query, however if you would have done:
Query:
```js
const user = Meteor.users.createQuery({
groupNames: 1,
groups: {
createdAt: 1,
}
}).fetchOne();
```
Result:
```
{
_id: 'XXX',
groupNames: ['Group 1', 'Group 2'],
groups: [
{_id: 'groupId1', createdAt: Date},
{_id: 'groupId2', createdAt: Date},
]
}
```
Notice that group `name` is not there. This is because we clean leftovers so the result is predictable.
## Reducers and reducers
You can also use other reducers inside your reducers.
```
// setting up
Users.addReducers({
fullName: {...}
fullNameWithRoles: { // the name of how you want to request it
body: { // the dependency, what info it needs to be able to reduce
fullName: 1,
roles: 1
},
reduce(object) { // a pure function that returns the data
return object.fullName + object.roles.join(',');
}
}
})
```
And again, unless you specified `fullName: 1` in your query, it will not be present in the result set.
## Params-aware reducers
By default the reducer receives the parameters the query has.
This can open the path to some nice customizations:
```js
Collection.addReducers({
reducer: {
body,
reduce(user, params) {}
}
})
```
Be aware that this reducer may be used from any queries with different types of parameters.
## Reducers can do anything!
If we want to just receive the number of posts a user has, we can use reducers for this:
```
Meteor.users.addReducers({
postCount: {
body: {_id: 1},
reduce(user) {
const linker = Users.getLink(user, 'posts');
return linker.find().count();
}
}
})
```
Or if you want to fetch some data from an external API:
Note that these reducers need to be defined server-side only, and they can only work with static queries.
```js
Projects.addReducers({
githubStars: {
body: {
repository: 1,
},
reduce(collectionItem) {
const {repository} = collectionItem;
const call = Meteor.wrapAsync(API.doSomething, API);
// you can use anything that is in sync
// don't return the result inside a callback because it won't work.
call();
},
}
})
```
## Filtering by reducers
If you want to filter reducers you can use `$postFilters` or `$postFilter` special functions.
## Conclusion
Reducers are a neat way to remove boilerplate from your code, especially for our infamous `emails[0].address`,
inside `Meteor.users` collection, check if you can figure out how to reduce it!

108
docs/structure_and_patterns.md Normal file
View file

@ -0,0 +1,108 @@
# Structure and Patterns
These are a set of recommended ways to handle things, they do not enforce
anything.
We suggest you read: http://www.meteor-tuts.com/chapters/3/persistence-layer.html first.
1. Store queries inside `/imports/api` under their own module and proper path.
2. Store `links` inside `/imports/db` along their collections definitions.
3. Create a an `/imports/db/index.js` that imports and exports all your collections
4. In `/imports/db/links.js` import all links from collections (`/imports/db/posts/links.js`)
5. In that `/imports/db/index.js` also `imports './links'` after you imported all collections.
6. Make sure you import `/imports/db/index.js` in both client and server environments.
7. For Named Queries, keep `query.js` and `query.expose.js` separated.
8. Create an `/imports/api/exposures.js` that imports all `.expose.js` files, and import that server-side.
8. When you import your queries suffix their with `Query`
9. Always `.clone()` queries before you use them client and server-side
10. Store reducers inside `links.js`, if the file becomes too large (> 100 lines), separate them.
If you respect the patterns above you will avoid having the most common pitfalls with Grapher:
**Reducer/Links not working?**
Make sure they are imported in the environment you use them client/server.
**My link is not saved in the database**
Make sure you added it correctly to your SimpleSchema
## Fragments
You will find yourself requiring often same fields for Users, such as email, fullName, and maybe avatar.
For that let's create some fragments:
```js
// file: /imports/db/fragments/UserPublic.js
export default {
fullName: 1,
avatar: {
path: 1,
},
email: 1,
}
// file: /imports/db/fragments/index.js
export {default as UserPublicFragment} from './UserPublicFields';
```
Now use it:
```js
import {UserPublicFragment} from '/imports/db/fragments';
Invoices.createQuery({
number: 1,
total: 1,
user: {
...UserPublicFragment,
billingInfo: {
// etc
}
}
})
```
You can also compose certain fragments:
```js
import compose from 'meteor/cultofcoders:grapher';
import {
UserPublicFragment,
UserBillingFragment,
} from '/imports/db/fragments';
Invoices.createQuery({
number: 1,
total: 1,
user: {
...compose(
UserPublicFragment,
UserBillingFragment
)
}
})
```
Compose uses a deep extension, so it works how you expected to work, especially if some fragments have shared bodies.
## Scaling Reactivity
If you want to have highly scalable reactive queries, think about moving from tailing MongoDB oplog to RedisOplog:
https://github.com/cult-of-coders/redis-oplog
Grapher is fully compatible with it. You can configure $options, inside the $filter() on to allow namespaced watchers.
Sample:
```js
export default Messages.createQuery('messagesForThread', {
$filter({filters, options, params}) {
filters.threadId = params.threadId;
options.namespace = `thread::${params.threadId}`;
},
text: 1,
})
```
## Conclusion
This ends our journey through Grapher. We hope you enjoyed, and that you are going to use it.

53
docs/table_of_contents.md Normal file
View file

@ -0,0 +1,53 @@
# Table of Contents
### [Introduction](introduction.md)
You are new to Grapher and you want to learn the basics of what it can offer.
### [Linking Collections](linking_collections.md)
Learn how to link collections to each other.
### [Linker Engine](linker_engine.md)
Find out what powers Grapher behind the scenes
### [Query Options](query_options.md)
Learn more advanced ways to use your queries.
### [Reducers](reducers.md)
Remove complexity from your queries, make them a breeze.
### [Named Queries](named_queries.md)
Learn how to define and expose queries to the client.
### [Hypernova](hypernova.md)
Read about another tool that powers Grapher behind the scenes.
### [Denormalization](denormalization.md)
Learn how to denormalize your data to enable even more performance, and super advanced searching.
### [Caching Results](caching_results.md)
Leverage the power of Grapher to cache the most heavy or frequently used queries.
### [Global Exposure](global_exposure.md)
Learn about a neat way to expose your collections, careful, it may also be dangerous.
### [Structure & Patterns](structure_and_patterns.md)
Learn about some good ways to structure your code some and about common pitfalls.
### [Outside Grapher](outside_grapher.md)
Use Grapher outside Meteor with ease.
### [API](api.md)
Grapher Cheatsheet.

4
main.client.js
View file

@ -17,3 +17,7 @@ export {
export {
default as NamedQuery
} from './lib/namedQuery/namedQuery.client';
export {
default as compose
} from './lib/compose';

8
main.server.js
View file

@ -23,3 +23,11 @@ export {
export {
default as MemoryResultCacher
} from './lib/namedQuery/cache/MemoryResultCacher';
export {
default as BaseResultCacher
} from './lib/namedQuery/cache/BaseResultCacher';
export {
default as compose
} from './lib/compose';