hiro/grapher
1
0
Fork 0
mirror of https://github.com/vale981/grapher synced 2025-03-05 09:31:42 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions

documentation rc

Browse source
This commit is contained in:
Theodor Diaconu 2017-12-01 12:30:09 +02:00
parent 7cc67a13f7
commit 0933b8c0b1
15 changed files with 450 additions and 365 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
README.md
View file

@ -2,23 +2,27 @@
[![Build Status](https://api.travis-ci.org/cult-of-coders/grapher.svg?branch=master)](https://travis-ci.org/cult-of-coders/grapher)
*Grapher* is a data retrieval layer inside Meteor and MongoDB. It's used in many production apps since 2016.
*Grapher* is a Data Fetching Layer on top of Meteor and MongoDB. It is production ready and battle tested.
Main features:
- Innovative way to make MongoDB relational
- Reactive data graphs for high availability
- Incredible performance
- Denormalization Modules
- Denormalization Ability
- Connection to external data sources
- Usable from anywhere
It marks a stepping stone into evolution of data, enabling developers to write complex and secure code,
while maintaining the code base easy to understand.
### [Documentation](docs/table_of_contents.md)
### [Documentation](docs/index.md)
This provides a learning curve for Grapher, explaining all the features.
### [API](docs/api.md)
Grapher cheatsheet, after you've learned it's powers this is the document will be very useful.
### Installation
```
meteor add cultofcoders:grapher

39
docs/caching_results.md
View file

@ -1,12 +1,11 @@
# Caching Results
Grapher is already performant enough to remove this necessity, but what if we want
to squeeze even more performance? What can we do? We cache'em up.
Ok, it's time for Grapher to take it to the next level. Let's cache our very heavy queries.
Caching results only works for Named Queries.
Let's say we have some very heavy queries, that do complex sorting
and complex filtering:
Let's take an example
```js
export default Meteor.users.createQuery('myFriendsEmails', {
@ -21,7 +20,7 @@ export default Meteor.users.createQuery('myFriendsEmails', {
```
Ok maybe it's not that complex or heavy, but use your imagination, imagine we have 100,000,000 users
inside the database. And you just want to look for your friends in this big world.
inside the database. And you just want to look for your friends in this big nasty world.
```js
// server-side
@ -35,7 +34,7 @@ myFriendsQuery.expose({
});
const cacher = new MemoryResultCacher({
ttl: 60 * 1000, // 1 minute caching
ttl: 60 * 1000, // 60 seconds caching
});
myFriendsQuery.cacheResults(cacher);
@ -44,18 +43,19 @@ myFriendsQuery.cacheResults(cacher);
That's it. If there is no cache, it fetches the database and stores the result in memory, and after 60s it clears the cache.
Any other calls done in between this time, will hit the cache.
The caching is done by serializing the params and prefixed by query name, in our case, the cache id looks like:
The cacher will be used regardless if you are on server or on the client.
The cacher also caches counts by default (eg: when you use `getCount()` from client or server)
**Cacher** is parameter bound, if you get the same parameters it will hit the cache (if it was already cached),
the cache id is generated like this:
```
myFriendsEmails{userId:"XXX"}
count::myFriendsEmails{userId:"XXX"}
```
The cacher will be used regardless if you are on server or on the client.
The cacher also works with counts. When you use `query.getCount()`
## Implementing your own
The cacher that you provide exposes:
The cacher needs to be an object which exposes:
```
fetch(cacheId, {
query: {fetch()},
@ -74,13 +74,17 @@ import {BaseResultCacher} from 'meteor/cultofcoders:grapher';
* Redis Cacher
*/
export default class RedisCacher extends BaseResultCacher {
// the constructor accepts a config object, that stores it in this.config
// this is the default one in case you need to override it
// if don't specify if it, it will use this one from BaseResultCacher
generateQueryId(queryName, params) {
return `${queryName}::${EJSON.stringify(params)}`;
}
// in case of a count cursor cacheId gets prefixed with 'count::'
fetch(cacheId, fetchables) {
// client and ttl are the configs passed when we instantiate it
const {client, ttl} = this.config;
const cacheData = client.get(cacheId);
@ -114,22 +118,21 @@ myFriendsQuery.cacheResults(cacher);
As you may have guessed, this works with resolver queries as well, in their case, instead of the actual query,
we pass as `query` parameter an interface containing `fetch()`.
And your cacher, will never receive `cursorCount` inside the `fetchables` object.
Therefore you can use the same paradigms for your cache for resolver queries as well.
And your cacher, will never receive `cursorCount` inside the `fetchables` object in this case.
Therefore you can use the same paradigms for your cache for resolver queries as well, without any change.
## Invalidation
Unfortunately, if you want to invalidate your cache you can do it yourself manually, as this is not implemented,
but since you can hook in your own cacher, you can do whatever you want.
## [Conclusion](table_of_contents.md)
## [Conclusion]
If you have very heavy and frequent requests to the database, or any type of requests (resolver queries) you can think
about using a cache.
about using a cache, with very few lines of code.
#### [Continue Reading](global_exposure.md) or [Back to Table of Contents](table_of_contents.md)

68
docs/denormalization.md
View file

@ -7,7 +7,8 @@ You may need denormalization in such cases where:
1. You want to avoid another database fetch
2. You need ability to perform complex filtering
This has some caveats that we'll explore, let's check out a simple example first:
A simple example, a user has an avatar that is stored in the image collection:
```js
// Assuming Images of schema: {_id, path, smallThumb, createdAt}
@ -46,10 +47,10 @@ const user = Meteor.users.createQuery({
}).fetchOne()
```
Grapher will check to see if avatar's body is a subbody of the denormalized body. If yes, it will hit the cache,
Grapher will check to see if avatar's body is a subbody of the denormalized body. If yes, it will hit the `avatarCache` field,
leading to a single database request.
And the result will be as you expect it to be:
And the result will in the form as you expect:
```
{
_id: 'XXX',
@ -72,46 +73,43 @@ const user = Meteor.users.createQuery({
}).fetchOne()
```
Will result in a subsequent database request, because `createdAt` is not in the denormalized body. But if you swap it with `path` then it will hit the cache.
Will result in a subsequent database request, because `createdAt` is not in the denormalized body. But if you replace `createdAt` with `path` then it will hit the cache.
When the user sets a new Avatar, or the `Image` object of that Avatar gets updated. The cache gets automatically updated,
so you don't have to worry about anything. It's magical.
Denormalization works with any type of links `one`, `many`, `meta` whether they are `direct` or `inversed`.
We previously tackled the case where we needed `$postFilters` or `$postFilter` to retrieve some special kind of data:
We previously tackled the case where we needed `$postFilters` or `$postFilter` to retrieve filtered data.
Let's take a case: we want to retrieve only the users that have reviewed a book of a certain type, and inside `books` collection,
we have `reviewedByUserIds`.
For example, let's say we want to retrieve only the users that have reviewed a book of a certain type,
and inside `users` collection we have a `reviewedBoos` link.
```js
Books.addLinks({
'reviewedByUsers': {
Meteor.users.addLinks({
'reviewedBooks': {
type: 'many',
collection: Meteor.users,
field: 'reviewedByUserIds',
collection: Books,
field: 'reviewedBooksIds',
denormalize: {
body: {
type: 1,
},
field: 'reviewedBooksCache',
}
}
})
Meteor.users.addLinks({
'bookReviews': {
collection: Books,
inversedBy: 'reviewedByUsers',
denormalize: {
body: {
type: 1,
},
field: 'bookReviewsCache',
}
Books.addLinks({
'reviewers': {
collection: Meteor.users,
inversedBy: 'reviewedBooks',
}
});
```
*Note that this case may not necessarily be the best example, this is just for illustration,
if you have a better example in mind, let us know so we can update this documentation.*
And now, I want to get all the users that have reviewed books of type `Drama`, because I want
to send them an email about a new book.
to send them an email about a new book, or a soap opera.
```js
const dramaticUsers = Meteor.users.createQuery({
@ -122,23 +120,34 @@ const dramaticUsers = Meteor.users.createQuery({
}).fetch();
```
Denormalization comes with a price:
That was it, but denormalization comes with a price:
1. It adds hooks to the database so it can properly update it, therefore a change somewhere can result
into additional computation.
2. If you are not careful it can lead to very big caches, which is not what you want unless you favor performance over storage.
The package also supports caching fields and caching counts. You can define those caches outside Grapher without a problem, and specify those fields in your query.
The `herteby:denormalize` package also supports caching fields and caching counts.
You can define those caches outside Grapher without a problem, and specify those fields in your query.
## Caution
If you want to use deep filters, it will not work with denormalized cache, you can use `$postFilter()` method for that.
```js
{
users: {
avatar: {
$filters: {} // will not hit the cache
}
}
}
```
Because if you put `$filters: {}` inside the body of the cache, it will regard it as a foreign field, and it will fetch the linked Collection for it.
A current limitation for denormalized meta links, is that we will no longer be able to store the `$metadata` inside the nested object, because that
would require additional fetching of the link storage,
would require additional fetching of the link storage if we are querying the graph from the inversed side.
## [Conclusion](table_of_contents.md)
## [Conclusion]
Using denormalization can enable you to do wonderful things inside NoSQL, but also be careful because they come with a cost,
that may not be very noticeable in the beginning. But can also dramatically improve performance at the same time.
@ -146,3 +155,4 @@ that may not be very noticeable in the beginning. But can also dramatically impr
I suggest that they should be used to cache things that rarely change such as an user's avatar, or when you need to do
powerful and frequent searches, that otherwise would have consumed more resources.
#### [Continue Reading](caching_results.md) or [Back to Table of Contents](table_of_contents.md)

11
docs/global_exposure.md
View file

@ -1,6 +1,7 @@
## Global Queries
Global queries are not recommended because they are very hard to secure.
Global queries are not recommended because they are very hard to secure. If you are not interested
in exposing an API for your clients or expose a public database, [continue reading next part](structure_and_patterns.md)
But they are very interesting in what they offer and they can prove to be very useful.
You can expose an API that has access to all or certain parts of your database, without
@ -8,7 +9,7 @@ defining a named query for each.
The difference between a `Named Query` and a `Global Query` is that the later
does not have their form defined on the server, the client can query for anything that he wishes
as long as the query is exposed and respects the security options.
as long as the query is exposed and respects the security restrictions.
A `Global Query` is as almost feature rich as a `Named Query` with the exception of caching.
@ -334,7 +335,9 @@ This will allow requests like:
}
```
## [Conclusion](table_of_contents.md)
## [Conclusion]
The global queries are a very powerful tool to expose your full database, but unlike `Named Queries` they do
not benefit of `caching`.
not benefit of `caching`.
#### [Continue Reading](structure_and_patterns.md) or [Back to Table of Contents](table_of_contents.md)

49
docs/hypernova.md
View file

@ -1,8 +1,10 @@
## Hypernova
This is the crown jewl of Grapher. It was named like this because it felt like an explosion of data.
This is the crown jewl of Grapher. It has been innovated in the laboratories of Cult of Coders,
and engineered for absolute performance. We had to name this whole process somehow, and we had
to give it a bombastic name. Hypernova is the one that stuck.
Grapher is very performant. To understand what we're talking about let's take this example of a query
To understand what we're talking about let's take this example of a query:
```js
createQuery({
@ -27,10 +29,10 @@ createQuery({
In a normal scenario, to retrieve this data graph we need to:
1. Fetch the posts
2. posts.length * Fetch categories for each post
3. posts.length * Fetch author for each post
4. posts.length * Fetch comments for each post
5. posts.length * 10 * Fetch author for each comment
2. length(posts) x Fetch categories for each post
3. length(posts) x Fetch author for each post
4. length(posts) x Fetch comments for each post
5. length(posts) * length(comments) * Fetch author for each comment
Assuming we have:
- 10 posts
@ -44,16 +46,16 @@ We would have blasted the database with:
- Categories: 10
- Post authors: 10
- Post comments: 10
- Post comments authors: 100
- Post comments authors: 10*10
This means `131` database requests.
This means 131 database requests.
Ok, you can cache some stuff, maybe some authors collide, but in order to write a performant code,
you would have to write a bunch of non-reusable code.
But this is just a simple query, imagine something deeper nested. Grapher simply destroys any other
MongoDB "relational" ORM.
But this is just a simple query, imagine something deeper nested. For Grapher, it's a breeze.
### Hypernova to the rescue
### Hypernova in Action
How many requests does the Hypernova?
- 1 for Posts
@ -63,15 +65,36 @@ How many requests does the Hypernova?
- 1 for all authors inside all comments
The number of database is predictable, because it represents the number of collection nodes inside the graph.
(If you use reducers that make use of links, take those into consideration as well)
It does this by aggregating filters and then it reassembles data locally.
It does this by aggregating filters at each level, fetching the data, and then it reassembles data to their
propper objects.
Not only it makes 5 requests instead of 131, but it smartly re-uses categories and authors at each collection node,
meaning you will have less bandwidth consumed.
Now you understand why this is a revolution for MongoDB.
Making it more efficient in terms of bandwidth than SQL. Yes, you read that correct:
```js
{
posts: {
categories: {
name: 1
}
}
}
```
Let's assume we have 100 posts, and the total number of categories is like 4. Hypernova does 2 requests to the database,
and fetches 100 posts, and 4 categories. If you would have used `JOIN` functionality in SQL, you would have received
categories for each post.
Now you understand why this is a revolution for MongoDB and Meteor.
Keep in mind that Hypernova is only used for static queries. For reactive queries, we still rely on the recursive fetching.
#### [Continue Reading](denormalization.md) or [Back to Table of Contents](table_of_contents.md)

57
docs/index.md Normal file
View file

@ -0,0 +1,57 @@
# Table of Contents
Let's learn Grapher. It's easy, we made a lot of effort to make it this way. It's time to change
the way you think about your data.
### [Introduction](introduction.md)
This introduces you to the way we use Grapher in the most basic form, without relationships.
### [Linking Collections](linking_collections.md)
Learn the various ways of linking collections to each other.
### [Linker Engine](linker_engine.md)
Learn about how you can programatically set and fetch related data.
### [Query Options](query_options.md)
Learn more advanced ways to use your queries.
### [Reducers](reducers.md)
Remove complexity from your queries, make them a breeze by using reducers.
### [Named Queries](named_queries.md)
Learn the right way to define and expose queries to the client in a secure manner.
### [Hypernova](hypernova.md)
Read about the tool that makes Grapher so performant.
### [Denormalization](denormalization.md)
Learn how to denormalize your data to enable even more performance, and super advanced searching.
### [Caching Results](caching_results.md)
Leverage the power of Grapher to cache the most heavy or frequently used queries.
### [Global Exposure](global_exposure.md)
Learn about a neat way to expose your collections. This was the initial approach of Grapher, and it is very useful
for when you want to expose an API accessing most parts of the database to your clients.
### [Structure & Patterns](structure_and_patterns.md)
Learn about some good ways to structure your code some and about common pitfalls.
### [Outside Meteor](outside_meteor.md)
If you want to use Grapher in a React Native app so from any other language as an API, it is possible
### [API](api.md)
After you've learned about Grapher, here's your cheatsheet.

64
docs/introduction.md
View file

@ -10,7 +10,7 @@ meteor add cultofcoders:grapher
Grapher is composed of 3 main modules, that work together:
### Link Manager
This module allows you to configure relationships between collections and allows you to create denormalized links and resolver links.
This module allows you to configure relationships between collections and allows you to create denormalized links.
### Query
The query module is used for fetching your data in a friendly manner, such as:
@ -22,7 +22,7 @@ createQuery({
})
```
It abstracts your query into a graph composed of Collection Nodes, Field Nodes and Resolver Nodes,
It abstracts your query into a graph composed of Collection Nodes and Field Nodes.
it uses the **Link Manager** to construct this graph and if the fetching is done server-side (non-reactive queries),
it uses the **Hypernova Module** the crown jewl of Grapher, which heavily minimizes requests to database.
@ -32,9 +32,9 @@ The exposure represents the layer between your queries and the client, allowing
only to users that have access.
### Your first query
### Let's begin!
You can use Grapher, without defining any links, for example, let's say you have a method which returns a list of posts.
You can use Grapher without defining any links, for example, let's say you have a method which returns a list of posts.
```js
Meteor.methods({
@ -50,7 +50,7 @@ Meteor.methods({
})
```
Transforming this into a Grapher query would simply look like this:
Transforming this into a Grapher query looks like this:
```js
Meteor.methods({
@ -71,8 +71,7 @@ you may find this cumbersome in the beginning, but as your application grows, ne
fields that need to be protected, you'll find yourself refactoring parts of your code-base which exposed
all the fields.
If, for example, you want to filter or sort your query, we have some special query variables to do so:
If, for example, you want to filter or sort your query, we introduce the `$filters` and `$options` fields:
```js
Meteor.methods({
@ -98,12 +97,16 @@ Meteor.methods({
If for example you are searching an element by `_id`, you may have `$filters: {_id: 'XXX'}`, then instead of `fetch()` you
can call `.fetchOne()` so it will return the first element found.
As you may have noticed, the $filters and $options are the ones you pass to `find()`.
`$filters` and `$options` are the ones supported by `Mongo.Collection.find()`
The nature of a Query is to be re-usable. For this we introduce a special type of field called `$filter`.
And we allow the query to receive parameters before it executes:
### Dynamic $filter()
The nature of a query is to be re-usable. For this we introduce a special type of field called `$filter`,
which allows the query to receive parameters and adapt before it executes:
```js
// We export the query, notice there is no .fetch()
export default Posts.createQuery({
$filter({filters, options, params}) {
filters.isApproved = params.isApproved;
@ -115,16 +118,15 @@ export default Posts.createQuery({
});
```
The `$filter()` function receives a single object that contains 3 objects: `filters`, `options`, `params`.
The `$filter()` function receives a single object composed of 3 objects: `filters`, `options`, `params`.
The `filters` and `options` are initially what you provided in `$filters` and `$options` query, they will be empty
if they haven't been specified.
objects if they haven't been specified.
The job of `$filter()` is to extend/modify `filters` and `options`, based on params.
Lets see how we can use that query:
Lets see how we can re-use the query defined above:
```js
// assuming you exported it from '...'
import postListQuery from '...';
Meteor.methods({
@ -136,11 +138,10 @@ Meteor.methods({
})
```
Whenever we want to use a modular query, we have to `clone()` it so it creates a new standalone instance,
that does not affect the exported one. The `clone()` accepts `params` as argument.
Those `params` will be passed to the `$filter` function.
Whenever we want to use a modular query, we have to `clone()` it so it creates a new instance of it.
The `clone()` accepts `params` as argument. Those `params` will be passed to the `$filter` function.
You could also use `setParams()` to configure parameters:
You can also use `setParams()` to configure parameters, which extends the current query parameters:
```js
import postListQuery from '...';
@ -149,6 +150,8 @@ Meteor.methods({
posts() {
const query = postListQuery.clone();
// Warning, if you don't use .clone() and you just .setParams(),
// those params will remain stored in your query
query.setParams({
isApproved: true,
});
@ -158,6 +161,8 @@ Meteor.methods({
})
```
### Validating Params
A query can be smart enough to know what parameters it needs, for this we can use the awesome `check` library from Meteor:
http://docs.meteor.com/api/check.html
@ -180,7 +185,8 @@ export default Posts.createQuery({
});
```
But you can craft your own validation:
If you want to craft your own validation, it also accepts a function that takes params:
```js
{
validateParams(params) {
@ -191,9 +197,9 @@ But you can craft your own validation:
}
```
Note: params validation is done prior to fetching the query.
Note: params validation is done prior to fetching the query, not when you do `setParams()` or `clone()`
And if you want to set some default parameters:
If you want to store some default parameters, you can use the `params` option:
```js
export default Posts.createQuery({...}, {
params: {
@ -202,19 +208,9 @@ export default Posts.createQuery({...}, {
});
```
## [Conclusion](table_of_contents.md)
## [Conclusion]
This is the end of our introduction. As we can see, we can make queries modular and this already gives us
a big benefit. By abstracting them into their own modules we can keep our methods neat and clean,
and we haven't even arrived to the good parts.
a big benefit. By abstracting them into their own modules we can keep our methods neat and clean.
#### [Continue Reading](linking_collections.md) or [Back to Table of Contents](table_of_contents.md)

49
docs/linker_engine.md
View file

@ -1,11 +1,11 @@
# Linker Engine
The linker engine is composed of 3 components:
- Definition of Links
- Linked Data Retrieval
- Setting Links
- **Definition of Links** (the `addLinks()` thingie, which we already covered)
- **Linked Data Retrieval** (fetching related linked data)
- **Setting Links** (actually updating the database)
Let's explore how we can play with `Linked Data Retrieval` and `Setting Links`, assumming we already defined our links
Let's explore how we can play with `Linked Data Retrieval` and `Setting Links`, after we already defined our links
using `addLinks()` method.
Each collection gets extended with a `getLink` function:
@ -30,13 +30,12 @@ const userGroupLinker = Meteor.users.getLink(userId, 'groups');
const groups = userGroupLinker.find().fetch();
// or for ease of use
const groups = userGroupLinker.fetch();
// or to filter the nested elements
// or to filter the linked elements
const groups = userGroupLinker.find(filters, options).fetch();
// and again simpler
const groups = userGroupLinker.fetch(filters, options);
// and if you want to performantly get a count()
// because find() returns a good ol' Mongo.Cursor
// and if you want to get a count()
const groups = userGroupLinker.find(filters, options).count();
```
@ -45,12 +44,10 @@ This works with any kind of links from any side.
## Setting Links
This allows you to very easily link collections to each other, without relying on knowing the fields and how are they stored.
It also allows you to set links from any place `direct` and `inversed`, of any type `one` or `many` and `meta` links as well:
It also allows you to set links from any place `direct` and `inversed`, of any type `one` or `many` and `meta` links as well,
enabling doing this in a natural way:
#### "One" Relationships
Performing a `set()` will automatically execute the update or insert in the database.
### One Links
```js
const userPaymentProfileLink = Meteor.users.getLink(userId, 'paymentProfile');
@ -64,7 +61,9 @@ const paymentProfileUserLink = PaymentProfiles.getLink(paymentProfileId, 'user')
paymentProfileUserLink.set(userId); // or a user object that contains `_id`
```
You can also `set` objects that aren't in the database yet:
You can also `set()` objects that aren't in the database yet.
Performing a `set()` will automatically execute the update or insert in the database.
```js
const userPaymentProfileLink = Meteor.users.getLink(userId, 'paymentProfile');
@ -78,12 +77,12 @@ This will insert into the `PaymentProfiles` collection and link it to user and i
To remove a link for a `one` relationship (no arguments required):
```js
// from direct or inversed side
userPaymentProfileLink.unset();
// or
paymentProfileUserLink.unset();
```
#### "Many" Relationships
### Many Links
Same principles as above apply, with some minor changes, this time we use `add` and `remove`
@ -106,19 +105,23 @@ userGroupsLink.add([
userGroupsLink.remove(groupIds)
```
The same logic applies, you can add array of objects that contain `_id` or array of objects without `_id`, or a mixture of them.
The same logic applies, you can:
- Single string *OR* Object with _id *OR* Object without _id
- Array of any mixture of the first ^
Ofcourse, the `remove()` cannot accept objects without `_id` as it makes no sense to do so.
The `remove()` cannot accept objects without `_id` as it makes no sense to do so.
#### "Meta" Relationships
### Meta Links
Now things get very interesting, because `metadata` allows us to store additional information about the link,
it lets us **describe** the relationship. And again, this works from `direct` and `inversed` side as well, with the
it lets us **describe** the relationship. This works from `direct` and `inversed` side as well, with the
same principles described as above.
The `add()` and `set()` allow an additional parameter `metadata`:
```js
// assumming our link now have {metadata: true} in their definition
// one
const userPaymentProfileLink = Meteor.users.getLink(userId, 'paymentProfile');
@ -159,12 +162,12 @@ userGroupsLink.metadata([groupId1, groupId2], {
Updating metadata only works with strings or objects that contain `_id`, and it works from both sides.
## [Conclusion](table_of_contents.md)
## [Conclusion]
By using this Programatic API to set your links instead of relying on updates, it makes your code much simpler to read,
much easier to migrate in the future to a new database relational model.
and makes schema migration easier in the future.
#### [Continue Reading](query_options.md) or [Back to Table of Contents](table_of_contents.md)

128
docs/linking_collections.md
View file

@ -1,14 +1,12 @@
# Linking Collections
Let's learn what types of links we can do between collections, and what is the best way to do them.
Let's learn what type of links we can define between collections, and what is the best way to do them.
First, we begin with an illustration of the power of Grapher:
First, we begin with an illustration of the power of Grapher.
Let's assume our posts, contain a field, called `authorId` which represents an actual `_id` from `Meteor.users`,
if you wanted to get the post and the author's name, you had to first fetch the post,
and then get the author's name based on his `_id`.
Let's assume our `posts` collection, contains a field, called `authorId` which represents an actual `_id` from `Meteor.users`,
if you wanted to get the post and the author's name, you had to first fetch the post, and then get the author's name based on his `_id`.
Something like:
```js
Meteor.methods({
getPost({postId}) {
@ -57,13 +55,14 @@ Meteor.methods({
This is just a simple illustration, imagine the scenario, in which you had comments,
and the comments had authors, and you needed their avatar. Your code can easily
grow to many lines of code, and it will be much less performant because of the many round-trips you do to
the database.
grow very large, and it's going to be hard to make it performant.
## Linking Types
## A basic link
To make the link illustrated in the example above, we create a separate `links.js` file that is
imported separately outside the collection module. We'll understand later why.
To define the link illustrated in the example above, we create a separate `links.js` file that is
imported separately outside the collection module. We need to define the links in their own module,
after all collections have been defined, because there will be situations where 2 collections import each other,
leading to some strange behaviors.
```js
// file: /imports/db/posts/links.js
@ -78,13 +77,13 @@ Posts.addLinks({
})
```
That was it. You created the link, and now you can use the query illustrated above.
We decided to choose `author` as a name for our link and `authorId` the field to store it in, but any string will do.
You created the link, and now you can use the query illustrated above.
We decided to choose `author` as a name for our link and `authorId` the field to store it in, but it's up to you to decide this.
### Inversed Links
## Inversed links
Because we linked `Posts` with `Meteor.users` it means that we can also get `posts` if we are in a User.
But because the link is stored in `Posts` we need a new type of linking, and we call it `Inversed Link`
Because we linked `Posts` with `Meteor.users` it means that we can also get all `posts` of an user.
Because in a way `Meteor.users` is also linked with `Posts` but an `inversed` way. We refer to it as an `Inversed Link`.
```js
// file: /imports/db/users/links.js
@ -98,7 +97,7 @@ Meteor.users.addLinks({
})
```
`author` represents the link name that was defined inside Posts. By defining inversed links we can do:
`author` represents the link name that was defined inside Posts. Defining inversed links allows us to do:
```js
Meteor.users.createQuery({
@ -108,19 +107,17 @@ Meteor.users.createQuery({
})
```
### One and Many
## One and Many
Above you've noticed a `type: 'one'` in the link definition, but let's say we have a `Post` that belongs to many `Categories`,
which have their own collection into the database. This means that we need to relate with more than a single element.
```js
// file: /imports/db/posts/links.js
import Posts from '...';
import Categories from '...';
Posts.addLinks({
'author': { ... },
'categories': {
type: 'many',
collection: Categories,
@ -129,9 +126,10 @@ Posts.addLinks({
})
```
In this case, `categoryIds` is an array of Strings, each String, representing `_id` from `Categories` collection.
In this case, `categoryIds` is an array of strings (`[categoryId1, categoryId2, ...]`), each string, representing `_id` from `Categories` collection.
Let's also create an inversed link from `Categories`, so you can use it inside the `query`
And, ofcourse, you can also create an inversed link from `Categories`, so you can use it inside the `query`
```js
// file: /imports/db/posts/links.js
import Categories from '...';
@ -145,10 +143,12 @@ Categories.addLinks({
})
```
By defining this, I can query for a category, and get all the posts it has.
## Meta Links
We use a `meta` link when we want to add additional data about the relationship. For example,
a user can belong in a single `Group`, but we need to know when he joined that group and what roles he has in it.
a user can belong in a single `Group`, but we need to also store when he joined that group and what roles he has in it.
```js
// file: /imports/db/users/links.js
@ -171,7 +171,7 @@ Notice the new option `metadata: true` this means that `groupLink` is no longer
{
...
groupLink: {
_id: 'XXX',
_id: 'XXX', // This is the _id of the group
roles: 'ADMIN',
createdAt: Date,
}
@ -204,7 +204,7 @@ const user = Meteor.users.createQuery({
}
```
We store the metadata of the link inside a special `$metadata` field. And this works from inversed side as well:
We store the metadata of the link inside a special `$metadata` field. And it works from inversed side as well:
```js
Groups.addLinks({
@ -213,7 +213,9 @@ Groups.addLinks({
inversedBy: 'group'
}
});
```
```js
const group = Groups.createQuery({
$filters: {_id: groupId},
name: 1,
@ -236,7 +238,8 @@ const group = Groups.createQuery({
},
_id: userId,
firstName: 'My Funky FirstName',
}
},
...
]
}
```
@ -253,24 +256,34 @@ The storage field will look like:
}
```
And they work the same as you expect, from the inversed side as well.
The same principles above apply, we still store `$metadata` field.
I know what question comes to your mind right now, what if I want to put a field inside the metadata,
I want to store who added that user (`addedBy`) to that link and fetch it, how do we do ?
which represents an id from other collection, like I want to store who added that user (`addedBy`) to the group.
Currently, this is not possible, because it has deep implications with how the Hypernova Module works,
but you can achieve this in a very performant way, by abstracting it into a separate collection:
No-one stops you from storing it as a string, but if this is one of Grapher's current limitations, it can't fetch
links that are inside the metadata.
But Grapher makes you think relational again, and you can abstract it to another collection:
```js
// file: /imports/db/groupUserLinks/links.js
import Groups from '...';
import GroupUserLinks from '...';
// file: /imports/db/users/links.js
Meteor.users.addLinks({
groupLink: {
collection: GroupUserLinks,
type: 'one',
field: 'groupLinkId',
}
})
GroupUserLinks.addLinks({
user: {
type: 'one',
collection: Meteor.users,
field: 'userId'
inversedBy: 'groupLink',
},
adder: {
type: 'one',
@ -283,15 +296,6 @@ GroupUserLinks.addLinks({
field: 'groupId'
}
})
// file: /imports/db/users/links.js
Meteor.users.addLinks({
groupLink: {
collection: GroupUserLinks,
type: 'one',
field: 'groupLinkId',
}
})
```
And the query will look like this:
@ -313,6 +317,7 @@ Meteor.users.createQuery({
## Link Loopback
No one stops you from linking a collection to itself, say you have a list of friends which are also users:
```js
Meteor.users.addLinks({
friends: {
@ -341,12 +346,15 @@ Meteor.users.createQuery({
## Uniqueness
The `type: 'one'` doesn't necessarily guarantee uniqueness from the inversed side.
For example, if we have inside `Comments` a link with `Posts` with `type: 'one'` inside postId,
it does not mean that when I fetch the posts with comments, comments will be an array.
The `type: 'one'` doesn't necessarily guarantee uniqueness from the inversed side.
But if you want to have a one to one relationship, and you want grapher to give you an object,
instead of an array you can do so:
For example, we have `Comments` and `Posts` linked, by defining a `one` link from Comments to Posts,
and an inversed link from Posts to Comments.
When you fetch comments, from posts, the inversed side, they will return an array.
But if you want to have a `OneToOne` relationship, and you want Grapher to give you a single object in return,
you can do:
```js
Meteor.users.addLinks({
@ -380,14 +388,16 @@ Meteor.users.createQuery({
## Data Consistency
This is referring to having consistency amongst links.
We clean out leftover links from deleted collection items.
Let's say I have a `thread` with multiple `members` from `Meteor.users`. If a `user` is deleted from the database, we don't want to keep unexisting references.
Let's say I have a `Threads` collection with some `memberIds` linked to `Meteor.users`.
If a `user` is deleted from the database, we don't want to keep unexisting references.
So after we delete a user, all threads containing that users should be cleaned.
This is done automatically by Grapher so you don't have to deal with it.
The only rule is that `Meteor.users` collection needs to have an inversed link to `Threads`.
The only rule is that `Meteor.users` collection needs to have an inversed link to `Threads`.
In conclusion, if you want to benefit of this, you have to define inversed links for every direct links.
## Autoremoval
@ -402,9 +412,12 @@ Meteor.users.addLinks({
});
```
After you deleted a user, all the links that have `autoremove: true` will be deleted.
After you delete a user, all the links that have `autoremove: true` will be deleted.
This works from the `direct` side as well, not only from `inversed` side.
This works from the `direct` side as well, not only from `inversed` side.
Please use with caution, sometimes it's better to explicitly delete it, but there will be situations,
where you don't care and this makes your code cleaner.
## Indexing
@ -424,24 +437,25 @@ PaymentProfiles.addLinks({
})
```
The index is applied only on the `_id`, meaning that if you have `meta` links, other fields present in that object will not be indexed.
The index is applied only on the `_id`, meaning that if you have `meta` links, other fields present in that object will not be indexed,
but you can run a `Collection._ensureIndex` separately.
If you have `unique: true` set, the index will also apply a unique constraint to it.
## Top Level Fields
Grapher currently supports only top level fields for storing data. One of the reasons it doesn't allow nested fields
is to enforce the developer to think relational and eliminate large and complex documents by abstracting them into collections.
Grapher currently supports only top level fields for storing linking data.
One of the reasons it doesn't allow nested fields is to enforce the developer to think relational and
eliminate large and complex documents by abstracting them into collections.
In the future, this limitation may change, but for now you can work around this and keep your code elegant.
## [Conclusion](table_of_contents.md)
## [Conclusion]
Using these simple techniques, you can create a beautiful database schemas inside MongoDB that are relational and very simple to fetch,
you will eliminate almost all your boilerplate code around this and allows you to focus on more important things.
On top of a cleaner code you benefit from the `Hypernova Module` which minimizes the database requests to a
predictable number (no. of collection nodes).
#### [Continue Reading](linker_engine.md) or [Back to Table of Contents](table_of_contents.md)

124
docs/named_queries.md
View file

@ -4,13 +4,13 @@ Before we explain what they are, we need to understand an alternative way of cre
## Alternative Creation
Currently, we only saw how to create a query starting from a collection, like:
Currently, we only know how to create a query starting from a collection, like:
```js
Meteor.users.createQuery(body, options);
```
But Grapher also exposes a `createQuery` functionality:
But Grapher also exposes a `createQuery` function:
```js
import {createQuery} from 'meteor/cultofcoders:grapher';
@ -22,7 +22,7 @@ createQuery({
})
```
The first key inside the object needs to represent an existing collection name:
The first key inside the object represents an existing collection name:
```js
const Posts = new Mongo.Collection('posts');
@ -36,10 +36,10 @@ createQuery({
## What are Named Queries ?
As the name implies, they are a query that are identified by a `name` (No shit Sherlock).
The difference is that they accept a `string` as the first argument.
As the name implies, they are a query that are identified by a `name`.
The difference is that they accept a `string` as the first argument, instead of a body.
The `Named Query` has the same API as a normal `Query`, we'll understand the difference in this documentation.
The `Named Query` has the same API and options as a normal `Query`, we'll understand the difference in this documentation.
```js
// file: /imports/api/users/queries/userAdminList.js
@ -63,13 +63,13 @@ Or you could use `createQuery`:
import {createQuery} from 'meteor/cultofcoders:grapher';
const admins = createQuery({
userAdminList: {},
userAdminList: params, // or {} if no params
}).fetch();
// will return a clone of the named query if it is already loaded
// will return a clone of the named query, with the params specified
```
Now notice the {}. Because `Named Queries` have their form well defined, they don't allow you
to specify fields directly, however they allow you to specify parameters when using `createQuery`
Because `Named Queries` have their form well defined, they don't allow you to specify fields directly,
however they allow you to specify parameters when using `createQuery`
Because the default `$filter()` allows as params `filters` and `options` for the top collection node,
you can do something like:
@ -82,27 +82,17 @@ const admins = createQuery({
}).fetch();
```
## Always go modular
## Go Modular, Always.
Ofcourse the recommended approach is to always use `modular`. Creating queries in files, and importing them accordingly.
The reason we expose such functionality is because if you want to use Grapher as an HTTP API, you do not have access
to the collections, so this becomes very handy, as you can transform a JSON request into a query.
Therefore, in such a case, a modular approach will not work.
The reason we expose such functionality is because if you want to use [Grapher as an HTTP API](outside_grapher.md),
you do not have access to the collections, so this becomes very handy, as you can transform a JSON request into a query.
In the client-side of Meteor it is recommended to always extract your queries into their own module, and import and clone them for use.
## But why ?
Why do we need another string? What is the purpose?
It's because we are slowly beginning to cross to the client-side domain, which
is a dangerous and nasty place and we need to expose our queries in a very cautious and secure manner.
## Special $body
`Named Queries` allow a special `$body` parameter. Let's see an advanced query:
`Named Queries` treat specially the `$body` parameter. Let's see an advanced query:
```js
const fullPostList = Posts.createQuery('fullPostList', {
@ -125,7 +115,7 @@ const fullPostList = Posts.createQuery('fullPostList', {
There will be situations where you initially want to show less data for this, maybe just the `title`, but if the user,
clicks on that `title` let's say we want it to expand and then we need to fetch additional data.
The way to do it is to use `$body` which intersects with the allowed data graph:
Instead of creating an additional `Named Query` with the same firewalls and such, you can use `$body` which intersects with the allowed data graph:
```js
fullPostsList.clone({
$body: {
@ -147,9 +137,7 @@ This will only fetch the intersection, the transformed body will look like:
}
```
When we learn about exposure you will fully understand why the `$body` special parameter is useful.
Be careful, if you use validation for params (and you should) to also add `$body` inside it:
Be careful, if you use validation for params (and you should) to also add `$body` inside it.
```js
import {Match} from 'meteor/check';
@ -162,15 +150,17 @@ const fullPostList = Posts.createQuery('fullPostList', {}, {
## Exposure
We are now crossing the bridge to client-side.
We are now crossing the bridge to client-side. It's a nasty place, filled with hacker, haxors, crackers,
we cannot trust them, we need to make the code unbreakable.
We want our application users to be able of using these queries we define stand-alone. We initially said that Grapher
will become the `Data Fetching Layer` inside Meteor. Therefore, we will no longer rely on methods to give us the data from
our queries, even if it's still possible and up to you to decide, but this gives you many advantages, such as caching and performance monitoring.
We initially said that Grapher will become the `Data Fetching Layer` inside Meteor.
Therefore, we will no longer rely on methods to give us the data from our queries, even if it's still possible
and up to you to decide, but this gives you many advantages, such as [Caching Results](caching_results.md) and performance monitoring
though Kadira/Meteor APM (because every named query exposed is a method and/or a publication)
As you may have guessed, the exposure needs to happen on the server, but the query can be imported on the client.
Let's say we want only Admins to be able to query our `userAdminList` query:
Let's say we want only users that are admins to be able to query our `userAdminList` query:
```js
// file: /imports/api/users/queries/userAdminList.js
@ -182,21 +172,9 @@ export default Meteor.users.createQuery('userAdminList', {
})
```
Alternatively, you could have done:
```js
// file: /imports/api/users/queries/userAdminList.js
export default createQuery('userAdminList', {
users: {
$filters: {
roles: {$in: 'ADMIN'}
},
name: 1,
}
})
```
```js
// server-side (make sure it's imported from somewhere on the server only)
// server-side
// make sure it's imported from somewhere on the server only
// file: /imports/api/users/queries/userAdminList.expose.js
import userAdminListQuery from './userAdminList';
@ -208,6 +186,9 @@ userAdminListQuery.expose({
// in the firewall you also have the ability to modify the parameters
// that are going to hit the $filter() function in the query
// the firewall runs in the Meteor.methods or Meteor.publish context
// Meaning you can have access to this.userId and others.
}
})
```
@ -219,9 +200,8 @@ Let's use it on the client side:
// client side
import userAdminListQuery from '/imports/api/users/queries/userAdminList.js';
userAdminListQuery.clone().fetch((err, res) => {
// do something with err, res
// it will be an error if the current user is not an admin
userAdminListQuery.clone().fetch((err, users) => {
// it will be an error if the current user is not an admin, thrown by the firewall
});
```
@ -253,7 +233,8 @@ userAdminListQuery.clone().fetchOne((err, user) => {
// do something
});
```
You can fetch static queries using promises:
You can use `fetchSync()` or `fetchOneSync()` to work with promises.
```js
const users = await userAdminListQuery.clone().fetchSync();
@ -269,6 +250,7 @@ query.getCount((err, count) => {
// do something
});
// promises
const count = await query.getCountSync();
// reactive counts
@ -282,25 +264,34 @@ Tracker.autorun(() => {
});
```
We do not subscribe to counts by default because they may be too expensive, but if you need them,
feel free to use them.
We do not subscribe to counts by default because they may be too expensive, especially for large collections,
but if you need them, feel free to use them, we worked hard at making them as performant as possible.
## Behind the scenes
When we are dealing with a static query (non-reactive), we make a call to the method that has been created when
we did `query.expose()`, the firewall gets applied and returns the result from the query.
we did `query.expose()`, params get checked, the firewall gets applied and returns the result from the query.
If we have a reactive query, `query.expose()` creates a publication end-point, and inside it, uses the `reywood:publish-composite` package
to automatically create the proper publication.
The method and publication names we create are: `named_query_${queryName}`, and as argument they accept the `params` object.
Meaning you can do:
- `Meteor.call('named_query_userAdminList', params, callback)`
- `Meteor.subscribe('named_query_userAdminList', params)`
But don't. Grapher takes care of this.
When we do `query.fetch()` on a reactive query, it gives you a complete data graph with the items in the same form as you
would have received from a static query. When an element inside your data graph changes, the `fetch()` function is re-run
if you are inside a `Tracker.autorun()` or a reactive context, because in the back, we do `find().fetch()` on client-side collections.
if you are inside a `Tracker.autorun()` or a reactive context, because in the back, it does `find().fetch()` on client-side collections,
and afterwards it assembles your data.
## Warning
If you want your client-side **reactive** queries to work as expected you need to make sure that the `addLinks` and `addReducers` are loaded on the client as well.
Because the data assembly is now done on the client.
If you want your client-side **reactive** queries to work as expected you need to make sure that the
`addLinks` and `addReducers` are loaded on the client as well, otherwise Grapher won't know how to assemble your data.
## Expose Options
@ -310,17 +301,22 @@ query.expose({
firewall(userId, params) {
// you can modify the parameters here
},
// Allow the query to be fetched statically
method: true, // default
// Allow the query to be fetched reactively (for heavy queries, you may want to set it to false)
publication: true, // default
// Unblocks your method (and if you have .unblock() in publication context it also unblocks it)
unblock: true, // default
// This can be an object or a function(params) that you don't want to expose it on the client via
// The query options as it may hold some secret business data
// If you don't specify it the default validateParams from the query applies, but not both!
// If you allow subbody requests, don't forget to add {$body: Match.Maybe(Boolean)}
validateParams: {},
// This deep extends your graph's body before processing it.
// For example, you want a hidden $filter() functionality, or anything else.
embody: {
@ -352,13 +348,15 @@ const getAnalytics = createQuery('getAnalytics', () => {}, {
});
```
Not the dummy `() => {}`. That's how we tell Grapher to make a resolver query.
```js
// server code only
import getAnalyticsQuery from './getAnalytics';
getAnalyticsQuery.expose({
firewall(userId, params) {
// ...
params.userId = userId;
},
validateParams: {} // Object or Function that you don't want exposed
});
@ -370,7 +368,9 @@ getAnalyticsQuery.resolve(function(params) {
```
The advantage of doing things like this, instead of relying on a method, is that you are now
using a uniform layer for fetching results, and on top of that, you can easily cache them.
using a uniform layer for fetching data, and on top of that, you can [easily cache them](caching_results.md).
If you try to `subscribe()` to a resolver query on the client it will throw an exception.
## Mix'em up
@ -391,8 +391,10 @@ query.expose({
})
```
## [Conclusion](table_of_contents.md)
## [Conclusion]
We can now safely expose our queries to the client, and the client can use it in a simple and uniform way.
By this stage we already understand how powerful Grapher really is, but it still has some tricks.
By this stage we already understand how powerful Grapher really is, but it still has some tricks up it's sleeve.
#### [Continue Reading](hypernova.md) or [Back to Table of Contents](table_of_contents.md)

38
docs/outside_grapher.md → docs/outside_meteor.md
View file

@ -1,13 +1,14 @@
## Grapher as an API
If you like Grapher, you can use it in any other language/medium.
If you like Grapher, or if `like` is an understatement, you can use it in any other language/medium.
You can expose it as an HTTP API, or a DDP API (as a Meteor method) for example, because
in React Native you have ways to connect to Meteor with:
You can expose it as an HTTP API, or a DDP API (as a Meteor.method()) for example,
because in ReactNative you have ways to connect to Meteor with:
https://www.npmjs.com/package/react-native-meteor#meteor-collections
Basically what Grapher needs to properly execute is the query,
and if you have firewalls, you need to manually handle authorization yourself.
Basically what Grapher needs to properly execute is the query, and you can make use of the firewalls easily,
but you need to manually handle authorization yourself.
### Exposing an HTTP API
@ -33,8 +34,7 @@ grapherRoutes.route('/grapher', function (req, res) {
const data = EJSON.parse(body);
// lets say this is a named query that looks like
// {getUserList: params}
const {query} = data;
const {query} = data; // query = {userList: params}
// authorize the user somehow
// it's up to you to extract an userId
@ -43,6 +43,8 @@ grapherRoutes.route('/grapher', function (req, res) {
const actualQuery = createQuery(query);
// if it's not a named query and the collection is not exposed, don't allow it.
// if you don't put this snippet of code, people will be able to do { users: { services: 1 } } types of queries.
// this is related to global queries.
if (actualQuery.isGlobalQuery && !actualQuery.collection.__isExposedForGrapher) {
throw new Meteor.Error('not-allowed');
}
@ -68,15 +70,25 @@ grapherRoutes.route('/grapher', function (req, res) {
})
```
Now you can do HTTP requests of `Content-Type: application/ejson` to http://meteor-server/grapher and retrieve data.
Now you can do HTTP requests of `Content-Type: application/ejson` to http://meteor-server/grapher and retrieve data,
in EJSON format. If `EJSON` is not available in your language, you can parse it with `JSON` as well.
If you want to use Meteor's Methods as an HTTP API to also handle method calls, take a look here:
- https://github.com/cult-of-coders/fusion
If you are in the JavaScript world: https://www.npmjs.com/package/ejson
If you want to use Meteor's Methods as an HTTP API to also handle method calls, take a look here: https://github.com/cult-of-coders/fusion
And more closely: https://github.com/cult-of-coders/fusion/blob/7ec5cd50c3a471c0bdd65c9fa482124c149dc243/fusion/server/route.js
### Exposing a DDP Method
If you are connected to Meteor by DDP, using any DDP Client:
- React Native: https://www.npmjs.com/package/react-native-meteor
- JS: https://www.npmjs.com/package/ddp-client
- PHP: https://github.com/zyzo/meteor-ddp-php
- Python: https://github.com/hharnisc/python-ddp
- Ruby: https://github.com/tmeasday/ruby-ddp-client
```js
import {createQuery} from 'meteor/cultofcoders:grapher';
@ -95,6 +107,8 @@ Meteor.methods({
})
```
## [Conclusion](table_of_contents.md)
## [Conclusion]
Nothing stops you from using Grapher outside Meteor!
#### [Continue Reading](api.md) or [Back to Table of Contents](table_of_contents.md)

60
docs/query_options.md
View file

@ -31,7 +31,7 @@ Now `user` will look like:
}
```
If you wanted to fetch the full `profile`, simply use `profile: 1` inside your query body. Alternatively,
If you want to fetch the full `profile`, use `profile: 1` inside your query body. Alternatively,
you can also use `'profile.firstName': 1` and `'profile.lastName': 1` but it's less elegant.
## Deep filtering
@ -41,13 +41,6 @@ Lets say we have a `Post` with `comments`:
```js
import {Comments, Posts} from '/imports/db';
Posts.addLinks({
comments: {
collection: Comments,
inversedBy: 'postId',
}
})
Comments.addLinks({
post: {
type: 'one',
@ -55,9 +48,16 @@ Comments.addLinks({
collection: Posts,
},
});
Posts.addLinks({
comments: {
collection: Comments,
inversedBy: 'postId',
}
})
```
If any bit of the code written above creates confusion, try reading again the `Linking Collections` part of the documentation.
If any bit of the code written above creates confusion, take another look on [Linking Collections](linking_collections.md).
We already know that we can query with `$filters`, `$options`, `$filter` and have some parameters.
The same logic applies for child collection nodes:
@ -74,9 +74,9 @@ Posts.createQuery({
})
```
The query above will fetch as `comments` only the ones that have been approved.
The query above will fetch as `comments` only the ones that have been approved and that are linekd with the `post`.
The `$filter` function share the same `params` across all collection nodes:
The `$filter` function shares the same `params` across all collection nodes:
```js
export default Posts.createQuery({
@ -145,7 +145,8 @@ Note the default $filter() only applies to the top collection node, otherwise we
### Pagination
There is a special field that extends the pre-fetch filtering process, and it's called `$paginate`:
There is a special field that extends the pre-fetch filtering process, and it's called `$paginate`, that allows us
to receive `limit` and `skip` params:
```js
const postsQuery = Posts.createQuery({
@ -166,13 +167,14 @@ const posts = postsQuery.clone({
}).fetch()
```
This is mostly used for your convenience, as pagination is a common used technique and makes your code easier to read.
This was created for your convenience, as pagination is a common used technique and makes your code easier to read.
Note that it doesn't override the $filter() function, it just applies `limit` and `skip` to the options, before $filter() runs.
Note that it doesn't override the $filter() function, it just applies `limit` and `skip` to the options, before `$filter()` runs.
It only works for the top level node, not for the child collection nodes.
### Meta Filters
Let's say we have `Users` that belong in `Groups` and they have some roles attached in the link description:
Let's say we have `users` that belong in `groups` and they have some roles attached in the link description:
```js
import {Groups} from '/imports/db';
@ -194,7 +196,7 @@ Groups.addLinks({
})
```
Let's assume the groupLinks looks like this:
Let's assume the `groupLinks` looks like this:
```js
[
{
@ -218,7 +220,8 @@ const users = Meteor.users.createQuery({
}).fetch()
```
But what if you want to fetch the groups and all their admins ? Easy.
But what if you want to fetch the groups and all their admins? It's the same.
```js
const groups = Groups.createQuery({
name: 1,
@ -236,7 +239,9 @@ We have gone through great efforts to support such functionality, but it makes o
## Post Filtering
This concept allows us to filter/manipulate data after we received it.
This concept allows us to filter/manipulate data after we retrived it and assembled it.
The `$postFilters` option uses the `sift` npm library (https://www.npmjs.com/package/sift) to make your filters look like MongoDB filters.
For example, what if you want to get the users that are admins in at least one group:
@ -252,15 +257,13 @@ const users = Meteor.users.createQuery({
}).fetch()
```
If you had a `many` relationship without `metadata` your $postFilters would look like:
If you had a `many` relationship without metadata your `$postFilters` would look like:
```
{
'groups.roles': {$in: 'ADMIN'},
}
```
The `$postFilters` option uses the `sift` npm library (https://www.npmjs.com/package/sift) to make your filters look like mongo filters.
In addition to `$postFilters` we've also got `$postOptions` that allows:
- limit
- sort
@ -302,10 +305,10 @@ const users = Meteor.users.createQuery({
}).fetch()
```
Note the fact that they only work for top level nodes, not for child collection nodes unlike `$filters` and `$options`.
Note the fact that these special fields only work for `top level nodes`, not for child collection nodes unlike `$filters`, `$options`, and `$filter`.
This type of queries that rely on post processing, can prove to be costly in some cases, because it will still fetch the users from the database
that don't have a group in which they are admin. There are alternatives to this to this in the `Denormalize` section of the documentation.
that don't have a group in which they are admin. There are alternatives to this to this in the [Denormalization](denormalization.md) section of the documentation.
It really depends on your context, but `$postFilters`, `$postOptions` and `$postFilter` can be very useful in some cases.
@ -317,11 +320,12 @@ If you want just to return the number of top level documents a query has:
query.getCount()
```
This will be very useful for pagination when we reach the client-side domain.
This will be very useful for pagination when we reach the client-side domain, or you just need a count.
Note that `getCount()` applies only the processed `filters` but not `options`.
## Mix'em up
The options `$filter` and `$postFilter` also allow you to provide an array of functions:
Both functions `$filter` and `$postFilter` also allow you to provide an array of functions:
```js
function userContext({filters, params}) {
@ -336,11 +340,10 @@ const posts = Posts.createQuery({
$filter: [userContext, ({filters, options, params}) => {
// do something
}],
$postFilter: [someOtherFunction],
}).fetch()
```
The example above is just to illustrate the possibility, in order to ensure that a `userId` param is sent you will use `validateParams`
The example above is just to illustrate the possibility, in order to ensure that a `userId` param is sent you will use `validateParams`.
```js
Posts.createQuery({
@ -365,11 +368,12 @@ const query = postLists.clone({
When we cross to the client-side domain we need to be very wary of these type of injections.
## [Conclusion](table_of_contents.md)
## [Conclusion]
Query is a very powerful tool, very flexible, it allows us to do very complex things that would have taken us a lot of time
to do otherwise.
#### [Continue Reading](reducers.md) or [Back to Table of Contents](table_of_contents.md)

30
docs/reducers.md
View file

@ -3,7 +3,7 @@
The reducers are a sort of "smart fields" which allow you to compose different results
from your query.
To achieve this we extend `Mongo.Collection` with an `addReducer()` method:
To achieve this we use inside `Mongo.Collection` the `addReducers()` method:
```js
Collection.addReducer({
reducerName: {
@ -50,7 +50,7 @@ Results to:
}
```
## Reducers and links
## Reducers that use links
Easily grab the data from your links (as deep as you want them), if you want to reduce it.
@ -61,7 +61,7 @@ Meteor.users.addReducers({
// assuming you have a link called groups
groups: { name: 1 }
},
reduce(object) { // a pure function that returns the data
reduce(object) {
return object.groups.map(group => group.name).join(',')
}
}
@ -110,7 +110,7 @@ Result:
Notice that group `name` is not there. This is because we clean leftovers so the result is predictable.
## Reducers and reducers
## Reducers can be composed
You can also use other reducers inside your reducers.
@ -118,13 +118,13 @@ You can also use other reducers inside your reducers.
// setting up
Users.addReducers({
fullName: {...}
fullNameWithRoles: { // the name of how you want to request it
body: { // the dependency, what info it needs to be able to reduce
fullNameWithRoles: {
body: {
fullName: 1,
roles: 1
},
reduce(object) { // a pure function that returns the data
return object.fullName + object.roles.join(',');
reduce(object) {
return object.fullName + ' ' + object.roles.join(',');
}
}
})
@ -148,9 +148,9 @@ Collection.addReducers({
Be aware that this reducer may be used from any queries with different types of parameters.
## Reducers can do anything!
## Reducers can be impure
If we want to just receive the number of posts a user has, we can use reducers for this:
If we want to just receive the number of posts a user posted, we can use reducers for this:
```
Meteor.users.addReducers({
@ -176,11 +176,11 @@ Projects.addReducers({
repository: 1,
},
reduce(collectionItem) {
const {repository} = collectionItem;
const call = Meteor.wrapAsync(API.doSomething, API);
// you can use anything that is in sync
// don't return the result inside a callback because it won't work.
call();
const {repository} = collectionItem;
const call = Meteor.wrapAsync(API.doSomething, API);
return call();
},
}
})
@ -190,7 +190,9 @@ Projects.addReducers({
If you want to filter reducers you can use `$postFilters` or `$postFilter` special functions.
## [Conclusion](table_of_contents.md)
## [Conclusion]
Reducers are a neat way to remove boilerplate from your code, especially for our infamous `emails[0].address`,
inside `Meteor.users` collection, check if you can figure out how to reduce it!
#### [Continue Reading](named_queries.md) or [Back to Table of Contents](table_of_contents.md)

35
docs/structure_and_patterns.md
View file

@ -5,17 +5,18 @@ anything.
We suggest you read: http://www.meteor-tuts.com/chapters/3/persistence-layer.html first.
1. Store queries inside `/imports/api` under their own module and proper path.
2. Store `links` inside `/imports/db` along their collections definitions.
3. Create a an `/imports/db/index.js` that imports and exports all your collections
4. In `/imports/db/links.js` import all links from collections (`/imports/db/posts/links.js`)
5. In that `/imports/db/index.js` also `imports './links'` after you imported all collections.
6. Make sure you import `/imports/db/index.js` in both client and server environments.
7. For Named Queries, keep `query.js` and `query.expose.js` separated.
8. Create an `/imports/api/exposures.js` that imports all `.expose.js` files, and import that server-side.
8. When you import your queries suffix their with `Query`
9. Always `.clone()` queries before you use them client and server-side
10. Store reducers inside `links.js`, if the file becomes too large (> 100 lines), separate them.
- Store queries inside `/imports/api` under their own module and proper path (eg: `/imports/api/users/queries/getAllUsers.js`)
- Store `links.js` files inside `/imports/db` close to their collections definitions. (eg: `/imports/db/users/links.js`)
- Create a an `/imports/db/index.js` that imports and exports all your collections
- In `/imports/db/links.js` which imports all links from collections (eg: `import ./posts/links.js`)
- In that `/imports/db/index.js` also `imports './links'` after you imported all collections.
- Make sure you import `/imports/db/index.js` in both client and server environments.
- For Named Queries, keep `query.js` and `query.expose.js` separated.
- Create an `/imports/api/exposures.js` that imports all `.expose.js` files, and import that server-side.
- When you import your queries suffix their with `Query`
- Always `.clone()` modular queries before you use them client and server-side
- Store reducers inside `links.js`, if the file becomes too large (> 100 lines), separate them.
- Store server-side reducers inside `/imports/api` - as they may contain business logic
If you respect the patterns above you will avoid having the most common pitfalls with Grapher:
@ -23,8 +24,7 @@ If you respect the patterns above you will avoid having the most common pitfalls
Make sure they are imported in the environment you use them client/server.
**My link is not saved in the database**
Make sure you added it correctly to your SimpleSchema
Make sure you added it correctly to your SimpleSchema, if you have that.
## Fragments
@ -83,9 +83,11 @@ Invoices.createQuery({
Compose uses a deep extension, so it works how you expected to work, especially if some fragments have shared bodies.
Do not use special properties inside Fragments, such as `$filters`, `$options`, etc.
## Scaling Reactivity
If you want to have highly scalable reactive queries, think about moving from tailing MongoDB oplog to RedisOplog:
If you want to have highly scalable reactive data graphs, think about moving from tailing MongoDB oplog to RedisOplog:
https://github.com/cult-of-coders/redis-oplog
Grapher is fully compatible with it. You can configure $options, inside the $filter() on to allow namespaced watchers.
@ -102,7 +104,8 @@ export default Messages.createQuery('messagesForThread', {
})
```
## [Conclusion]
## [Conclusion](table_of_contents.md)
Using some simple techniques we can make our code much easier to read, and we can make use of a scalable data graph using `redis-oplog`
This ends our journey through Grapher. We hope you enjoyed, and that you are going to use it.
#### [Continue Reading](outside_meteor.md) or [Back to Table of Contents](table_of_contents.md)

53
docs/table_of_contents.md
View file

@ -1,53 +0,0 @@
# Table of Contents
### [Introduction](introduction.md)
You are new to Grapher and you want to learn the basics of what it can offer.
### [Linking Collections](linking_collections.md)
Learn how to link collections to each other.
### [Linker Engine](linker_engine.md)
Find out what powers Grapher behind the scenes
### [Query Options](query_options.md)
Learn more advanced ways to use your queries.
### [Reducers](reducers.md)
Remove complexity from your queries, make them a breeze.
### [Named Queries](named_queries.md)
Learn how to define and expose queries to the client.
### [Hypernova](hypernova.md)
Read about another tool that powers Grapher behind the scenes.
### [Denormalization](denormalization.md)
Learn how to denormalize your data to enable even more performance, and super advanced searching.
### [Caching Results](caching_results.md)
Leverage the power of Grapher to cache the most heavy or frequently used queries.
### [Global Exposure](global_exposure.md)
Learn about a neat way to expose your collections, careful, it may also be dangerous.
### [Structure & Patterns](structure_and_patterns.md)
Learn about some good ways to structure your code some and about common pitfalls.
### [Outside Grapher](outside_grapher.md)
Use Grapher outside Meteor with ease.
### [API](api.md)
Grapher Cheatsheet.