MongoDB


Tuning

MongoDB tuning involves a holistic approach, focusing on data modeling (embedding vs. referencing), indexing (creating efficient indexes for queries), query optimization (using explain() for slow queries, projection), server configuration (memory, storage, concurrency settings like tickets, connection pooling), and monitoring (profiler, Atlas tools) to ensure optimal performance, scalability, and resource usage for your specific application needs. 
This video provides an overview of the MongoDB architecture for query performance:

Key Areas for Tuning
  1. Data Modeling & Indexing:
    • Embed vs. Reference: Model data for common access patterns, embedding related data for fewer reads, but referencing when data is shared or grows large (minimizing document size).
    • Index Strategically: Create indexes on fields used in find()sort(), and aggregate() stages. Use compound indexes for multi-field queries.
    • Avoid Collection Scans: Use explain() to ensure queries use index scans, not full collection scans. 
This video explains the importance of indexes for performance:
  1. Query & Aggregation Optimization:
    • Use the Profiler: Enable the database profiler to find slow queries exceeding a threshold.
    • explain() Plan: Analyze explain() output to see if indexes are used, how many documents are examined, and identify bottlenecks.
    • Projection: Use projection to return only needed fields, reducing network traffic and memory.
    • Aggregation Pipelines: Optimize stages, push filters down, and use appropriate operators for complex data processing. 
This video demonstrates how to use the profiler and analyze query plans:
  1. Server & Storage Engine Tuning:
    • WiredTiger Tickets: Adjust wiredTigerTicketValues for read/write concurrency if operations queue up (tickets hit 0).
    • Memory: Configure storage.wiredTiger.engineConfig.cacheSizeGB for optimal WiredTiger cache size.
    • Compression: Enable compression (e.g., Snappy, zlib) to reduce I/O and storage, often improving performance. 
This video discusses memory settings and their impact on performance:
  1. Connection Management:
    • Connection Pooling: Tune minPoolSizemaxPoolSize, and socketTimeoutMS in your drivers to match application load and network conditions. 
This video explores patterns for tuning MongoDB performance and scalability:
  1. Hardware & OS (Advanced):
    • NUMA Settings: Configure BIOS/OS settings (like iommu=pt) for optimal CPU/memory interaction on NUMA systems. 

General Approach

MongoDB: average value of field

To calculate the average value of a field in MongoDB using the shell, the aggregation framework with the $avg operator is utilized.
Example 1: Calculating the average of a field across the entire collection
Assume a collection named products with documents like:
Code
{ "_id": 1, "item": "A", "price": 10 }
{ "_id": 2, "item": "B", "price": 15 }
{ "_id": 3, "item": "C", "price": 20 }
{ "_id": 4, "item": "D", "price": 12 }
To find the average price of all products:
JavaScript
db.products.aggregate([
  {
    $group: {
      _id: null, // Group all documents into a single group
      averagePrice: { $avg: "$price" } // Calculate the average of the 'price' field
    }
  }
])
This query would return a result similar to:
Code
{ "_id": null, "averagePrice": 14.25 }
Example 2: Calculating the average of a field grouped by another field
To find the average price grouped by item:
JavaScript
db.products.aggregate([
  {
    $group: {
      _id: "$item", // Group documents by the 'item' field
      averagePrice: { $avg: "$price" } // Calculate the average of 'price' for each group
    }
  }
])
If the products collection contains:
Code
{ "_id": 1, "item": "A", "price": 10 }
{ "_id": 2, "item": "B", "price": 15 }
{ "_id": 3, "item": "A", "price": 20 }
{ "_id": 4, "item": "B", "price": 12 }
This query would return a result similar to:
Code
[
  { "_id": "B", "averagePrice": 13.5 },
  { "_id": "A", "averagePrice": 15 }
]

MongoDB : sorting documents by multiple fields

To sort documents by multiple fields in the MongoDB shell, use the sort() method after your find() query. The sort() method accepts a document where keys are the field names and values are either 1 for ascending order or -1 for descending order. The order of fields within the sort() document determines the sorting priority.
Here's an example:
Assume a collection named students with documents like:
Code
{ "name": "Alice", "grade": "A", "age": 18 }
{ "name": "Bob", "grade": "B", "age": 19 }
{ "name": "Charlie", "grade": "A", "age": 17 }
{ "name": "David", "grade": "B", "age": 18 }
To sort these documents first by grade in ascending order, and then by age in descending order for students with the same grade:
JavaScript
db.students.find().sort({ grade: 1, age: -1 });
Explanation:
  • db.students.find(): This retrieves all documents from the students collection.
  • .sort({ grade: 1, age: -1 }): This applies the sorting criteria.
    • grade: 1: Documents are primarily sorted by the grade field in ascending order (e.g., "A" before "B").
    • age: -1: For documents that have the same grade, they are then sorted by the age field in descending order (e.g., 19 before 18).
Example Output (based on the sample data):
Code
{ "name": "Charlie", "grade": "A", "age": 17 }
{ "name": "Alice", "grade": "A", "age": 18 }
{ "name": "Bob", "grade": "B", "age": 19 }
{ "name": "David", "grade": "B", "age": 18 }

MongoDB: shell commands examples

Here are examples of common operations within the MongoDB Shell (mongosh):
1. Database Management:
show databases.
JavaScript
    show dbs
  • Switch to a database (or create if it doesn't exist):
JavaScript
    use myDatabase
Show current database.
JavaScript
    db
2. Collection Management:
Show collections in the current database.
JavaScript
    show collections
Create a new collection.
JavaScript
    db.createCollection("myCollection")
3. Data Manipulation (CRUD Operations):
Insert a single document.
JavaScript
    db.myCollection.insertOne({ name: "Alice", age: 30, city: "New York" })
Insert multiple documents.
JavaScript
    db.myCollection.insertMany([
        { name: "Bob", age: 25, city: "London" },
        { name: "Charlie", age: 35, city: "Paris" }
    ])
Find all documents in a collection.
JavaScript
    db.myCollection.find()
Find documents with specific criteria.
JavaScript
    db.myCollection.find({ age: { $gt: 28 } }) // Find documents where age is greater than 28
Find documents and project specific fields.
JavaScript
    db.myCollection.find({}, { name: 1, city: 1, _id: 0 }) // Include name and city, exclude _id
Update a single document.
JavaScript
    db.myCollection.updateOne({ name: "Alice" }, { $set: { age: 31 } })
Update multiple documents.
JavaScript
    db.myCollection.updateMany({ city: "London" }, { $set: { status: "active" } })
Delete a single document.
JavaScript
    db.myCollection.deleteOne({ name: "Bob" })
Delete multiple documents.
JavaScript
    db.myCollection.deleteMany({ status: "inactive" })
4. Utility Commands:
Clear the shell screen.
JavaScript
    cls
exit the shell.
JavaScript
    exit

MongoDB: find documents matching regular expression

To find documents matching a regular expression in the MongoDB shell, use the find() method with the $regex operator or a regex literal.
Using a Regex Literal:
JavaScript
db.collectionName.find({ fieldName: /pattern/options });
Example: Find documents in the users collection where the name field contains "john" (case-insensitive). 
JavaScript
db.users.find({ name: /john/i });
Using the $regex Operator:
JavaScript
db.collectionName.find({ fieldName: { $regex: "pattern", $options: "options" } });
Example: Find documents in the products collection where the description field contains "eco-friendly" (case-insensitive).
JavaScript
db.products.find({ description: { $regex: "eco-friendly", $options: "i" } });
Explanation:
  • collectionName: The name of the collection to query.
  • fieldName: The field within the documents to apply the regular expression to.
  • pattern: The regular expression pattern to match.
  • options: Optional flags to modify the regex behavior. Common options include:
    • i: Case-insensitive matching.
    • m: Multi-line matching.
    • s: Allows the dot (.) to match newline characters.
    • x: Ignores whitespace characters in the pattern unless escaped.