MongoDB
- Tuning
- MongoDB: average value of field
- MongoDB : sorting documents by multiple fields
- MongoDB: shell commands examples
- MongoDB: find documents matching regular expression
Tuning
MongoDB tuning involves a holistic approach, focusing on data modeling (embedding vs. referencing), indexing (creating efficient indexes for queries), query optimization (using
explain() for slow queries, projection), server configuration (memory, storage, concurrency settings like tickets, connection pooling), and monitoring (profiler, Atlas tools) to ensure optimal performance, scalability, and resource usage for your specific application needs. This video provides an overview of the MongoDB architecture for query performance:
Key Areas for Tuning
- Data Modeling & Indexing:
- Embed vs. Reference: Model data for common access patterns, embedding related data for fewer reads, but referencing when data is shared or grows large (minimizing document size).
- Index Strategically: Create indexes on fields used in
find(),sort(), andaggregate()stages. Use compound indexes for multi-field queries. - Avoid Collection Scans: Use
explain()to ensure queries use index scans, not full collection scans.
This video explains the importance of indexes for performance:
- Query & Aggregation Optimization:
- Use the Profiler: Enable the database profiler to find slow queries exceeding a threshold.
explain()Plan: Analyzeexplain()output to see if indexes are used, how many documents are examined, and identify bottlenecks.- Projection: Use
projectionto return only needed fields, reducing network traffic and memory. - Aggregation Pipelines: Optimize stages, push filters down, and use appropriate operators for complex data processing.
This video demonstrates how to use the profiler and analyze query plans:
- Server & Storage Engine Tuning:
- WiredTiger Tickets: Adjust
wiredTigerTicketValuesfor read/write concurrency if operations queue up (tickets hit 0). - Memory: Configure
storage.wiredTiger.engineConfig.cacheSizeGBfor optimal WiredTiger cache size. - Compression: Enable compression (e.g., Snappy, zlib) to reduce I/O and storage, often improving performance.
- WiredTiger Tickets: Adjust
This video discusses memory settings and their impact on performance:
- Connection Management:
- Connection Pooling: Tune
minPoolSize,maxPoolSize, andsocketTimeoutMSin your drivers to match application load and network conditions.
- Connection Pooling: Tune
This video explores patterns for tuning MongoDB performance and scalability:
- Hardware & OS (Advanced):
- NUMA Settings: Configure BIOS/OS settings (like
iommu=pt) for optimal CPU/memory interaction on NUMA systems.
- NUMA Settings: Configure BIOS/OS settings (like
General Approach
- Monitor: Use MongoDB Atlas metrics or tools like
mongostat,mongotop, and the profiler. - Methodical Changes: Apply changes one at a time and measure the impact.
- Balance: Indexing speeds up reads but slows writes; find the right balance for your workload.
MongoDB: average value of field
To calculate the average value of a field in MongoDB using the shell, the aggregation framework with the
$avg operator is utilized.Example 1: Calculating the average of a field across the entire collection
Assume a collection named
products with documents like:{ "_id": 1, "item": "A", "price": 10 }
{ "_id": 2, "item": "B", "price": 15 }
{ "_id": 3, "item": "C", "price": 20 }
{ "_id": 4, "item": "D", "price": 12 }
To find the average
price of all products:db.products.aggregate([
{
$group: {
_id: null, // Group all documents into a single group
averagePrice: { $avg: "$price" } // Calculate the average of the 'price' field
}
}
])
This query would return a result similar to:
{ "_id": null, "averagePrice": 14.25 }
Example 2: Calculating the average of a field grouped by another field
To find the average
price grouped by item:db.products.aggregate([
{
$group: {
_id: "$item", // Group documents by the 'item' field
averagePrice: { $avg: "$price" } // Calculate the average of 'price' for each group
}
}
])
If the
products collection contains:{ "_id": 1, "item": "A", "price": 10 }
{ "_id": 2, "item": "B", "price": 15 }
{ "_id": 3, "item": "A", "price": 20 }
{ "_id": 4, "item": "B", "price": 12 }
This query would return a result similar to:
[
{ "_id": "B", "averagePrice": 13.5 },
{ "_id": "A", "averagePrice": 15 }
]
MongoDB : sorting documents by multiple fields
To sort documents by multiple fields in the MongoDB shell, use the
sort() method after your find() query. The sort() method accepts a document where keys are the field names and values are either 1 for ascending order or -1 for descending order. The order of fields within the sort() document determines the sorting priority.Here's an example:
Assume a collection named
students with documents like:{ "name": "Alice", "grade": "A", "age": 18 }
{ "name": "Bob", "grade": "B", "age": 19 }
{ "name": "Charlie", "grade": "A", "age": 17 }
{ "name": "David", "grade": "B", "age": 18 }
To sort these documents first by
grade in ascending order, and then by age in descending order for students with the same grade:db.students.find().sort({ grade: 1, age: -1 });
Explanation:
db.students.find(): This retrieves all documents from thestudentscollection..sort({ grade: 1, age: -1 }): This applies the sorting criteria.grade: 1: Documents are primarily sorted by thegradefield in ascending order (e.g., "A" before "B").age: -1: For documents that have the samegrade, they are then sorted by theagefield in descending order (e.g., 19 before 18).
Example Output (based on the sample data):
{ "name": "Charlie", "grade": "A", "age": 17 }
{ "name": "Alice", "grade": "A", "age": 18 }
{ "name": "Bob", "grade": "B", "age": 19 }
{ "name": "David", "grade": "B", "age": 18 }
MongoDB: shell commands examples
Here are examples of common operations within the MongoDB Shell (mongosh):
1. Database Management:
show databases.
show dbs
- Switch to a database (or create if it doesn't exist):
use myDatabase
Show current database.
db
2. Collection Management:
Show collections in the current database.
show collections
Create a new collection.
db.createCollection("myCollection")
3. Data Manipulation (CRUD Operations):
Insert a single document.
db.myCollection.insertOne({ name: "Alice", age: 30, city: "New York" })
Insert multiple documents.
db.myCollection.insertMany([
{ name: "Bob", age: 25, city: "London" },
{ name: "Charlie", age: 35, city: "Paris" }
])
Find all documents in a collection.
db.myCollection.find()
Find documents with specific criteria.
db.myCollection.find({ age: { $gt: 28 } }) // Find documents where age is greater than 28
Find documents and project specific fields.
db.myCollection.find({}, { name: 1, city: 1, _id: 0 }) // Include name and city, exclude _id
Update a single document.
db.myCollection.updateOne({ name: "Alice" }, { $set: { age: 31 } })
Update multiple documents.
db.myCollection.updateMany({ city: "London" }, { $set: { status: "active" } })
Delete a single document.
db.myCollection.deleteOne({ name: "Bob" })
Delete multiple documents.
db.myCollection.deleteMany({ status: "inactive" })
4. Utility Commands:
Clear the shell screen.
cls
exit the shell.
exit
MongoDB: find documents matching regular expression
To find documents matching a regular expression in the MongoDB shell, use the
find() method with the $regex operator or a regex literal.Using a Regex Literal:
db.collectionName.find({ fieldName: /pattern/options });
Example: Find documents in the
users collection where the name field contains "john" (case-insensitive).
db.users.find({ name: /john/i });
Using the
$regex Operator:db.collectionName.find({ fieldName: { $regex: "pattern", $options: "options" } });
Example: Find documents in the
products collection where the description field contains "eco-friendly" (case-insensitive).db.products.find({ description: { $regex: "eco-friendly", $options: "i" } });
Explanation:
collectionName: The name of the collection to query.fieldName: The field within the documents to apply the regular expression to.pattern: The regular expression pattern to match.options: Optional flags to modify the regex behavior. Common options include:i: Case-insensitive matching.m: Multi-line matching.s: Allows the dot (.) to match newline characters.x: Ignores whitespace characters in the pattern unless escaped.