Skip to content

Commit 1b4c4f5

Browse files
oswaldo-osunaoswaldo-osuna
committed
2 parents 8a14593 + fa2f3f5 commit 1b4c4f5

8 files changed

+604
-5
lines changed

‎oci-opentelemetry-solution/README.md‎

Lines changed: 140 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@ The included files will allow you to monitor your MERN application with:
1111

1212

1313

14-
1514
# Step 1: Create an APM Domain in OCI
1615
Follow the documentation to create your domain
1716
https://docs.oracle.com/en-us/iaas/application-performance-monitoring/doc/create-apm-domain.html
@@ -33,6 +32,78 @@ Take note of the follwoing:
3332
<img width="1507" alt="Screenshot 2023-10-10 at 3 01 11 PM" src="https://github.com/zkhader/oci-observability-and-management/assets/14898804/58a7aa34-ab28-48a2-80b0-65f2a65745e0">
3433

3534

35+
# Step 2: Create Logging Anaytics configuration
36+
37+
1. Create API signing key
38+
* Login to OCI console → User → User settings
39+
* Select API Keys under Resources
40+
* Click on Add API Key → Generate API Key Pair → Click on Download Private Key & then Add
41+
* Copy the content from Configuration File Preview & Close
42+
43+
<br>
44+
<img width="720" alt="1 UserProfile" src="https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/160bb472-c96a-4727-970f-8826f19a7827">
45+
46+
2. Create a Configuration File
47+
* Create a directory .oci and a config file with content from configuration file preview and path to private key file
48+
49+
```
50+
config
51+
[DEFAULT]
52+
user=ocid1.user.oc1..aaaaaaaamfy2qbkj7udfwmep34njicnu6skbzuddb52w4v4d7r2oepa3x5ya
53+
fingerprint=8a:ff:62:12:05:c8:29:80:4a:3d:0b:54:ac:86:85:e2
54+
tenancy=ocid1.tenancy.oc1..aaaaaaaa5s2vdjjrydixjulorcwozffbpna37w5a35p3jhgpyshlkmio6oiq
55+
region=us-ashburn-1
56+
key_file=~/.oci/oci-priv-key.pem
57+
```
58+
59+
3. Create a Log Parser
60+
* Observability & Management → Logging Analytics → Administration
61+
* Click on Parsers → Create Parser → Choose JSON type
62+
* Enter example JSON log content it will parse and extract fields and map it to specific field names as needed and click save changes.
63+
64+
LogRecord
65+
```
66+
{"name":"OCILogger","hostname":"emcc.marketplace.com","pid":12586,"level":50,"msg":"Inside delete method:Cannot Delete with id 649b4ac1883092297279051b. Maybe id
67+
is wrong","time":"2023-07-10T07:02:13.678Z","src":{},"v":0,"trace_id":"5941ccd308fcb49e30b3ebfcffcff38f","span_id":"0bed4a8c3b33d48e","trace_flags":"01"}
68+
```
69+
<br>
70+
<img width="1285" alt="2 Parser" src="https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/ebaaa401-bc7e-4e6b-a220-7b9a441bddcc">
71+
72+
4. Create a Log Source
73+
* Logging Analytics → Administration → Sources → Create Source
74+
* Source Type → File
75+
* Entity Types → Host (Linux)
76+
* Select Specific Parser → Select the parser created
77+
* Click on Create Source
78+
79+
<br>
80+
<img width="1260" alt="3 LogSource" src="https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/f967a34d-3835-4126-a5c8-7ffe8f1b267b">
81+
82+
83+
5. Create a Log Group
84+
* Logging Analytics → Administration → Log Groups → Create Log Group
85+
* Provide Log Group Name and description → click create
86+
87+
<br>
88+
<img width="789" alt="4 LogGroup" src="https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/32cbdd5f-ce3b-4bb3-bce1-96e6e2a4973e">
89+
<br>
90+
> Make a note of OCID of Log Group which will be used later
91+
<br>
92+
<img width="1099" alt="5 LogGroup-2" src="https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/8fa7770e-a9a5-44a5-acd7-c6882085a46f">
93+
94+
95+
6. Get the Namespace Details
96+
* Goto Identity → Compartments → Click on compartment where the log source is created → Copy OCID of the compartment
97+
<br>
98+
<img width="998" alt="6 Compartment-OCID-2" src="https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/06b6761e-a0d1-4312-afcb-d514d2a2343c">
99+
100+
101+
* Now launch cloud shell and execute below command to get the namespace
102+
```
103+
oci os ns get -c compartmentID
104+
```
105+
<br>
106+
<img width="1231" alt="7 Namespace" src="https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/4c28b966-fe94-477a-97ba-5708a4ea5ecb">
36107

37108

38109
# Step 2: Add the files from the repo to your project
@@ -53,6 +124,44 @@ tracing.js file:
53124

54125
<img width="592" alt="Screenshot 2023-10-10 at 3 17 26 PM" src="https://github.com/zkhader/oci-observability-and-management/assets/14898804/08116128-2cd1-490f-a8f6-8439e0e8875f">
55126

127+
ocilogginganalytics.js file:
128+
- Edit the file to update the parameters mentioned below
129+
[PATH]/config - Path to config file
130+
[PROFILE] - Profile in config file to be used for OCI authentication
131+
[NAMESPACE] - Namespace
132+
[UPLOADNAME] - User defined name for uploads
133+
[LOGSOURCENAME] - Log Source Name created in OCI logging analytics
134+
[LOGGROUPID] - Log Group ID created in Logging analytics to group the log messages
135+
[BUFFERLENGTH] - Buffer size (number of log messages to store)
136+
[FLUSHINTERVAL] – Flush internal in milliseconds to flush messages from buffer and send it to OCI LA
137+
[LOGGERNAME] – user defined name to initialize the bunyan logger
138+
139+
<br>
140+
<img width="1116" alt="image" src="https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/cca9667b-caa8-46fb-af68-09e9a5ecfd7e">
141+
<br>
142+
<img width="1056" alt="image" src="https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/bbf57fd6-32d8-424b-85ca-eeab8f745d82">
143+
<br>
144+
<img width="973" alt="image" src="https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/f5355466-1864-46ac-a394-6f4d767f9b65">
145+
<br>
146+
<img width="986" alt="image" src="https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/d1143df6-dc28-40cc-b067-3bab0ba7eafe">
147+
148+
149+
- Modify the application source files to initialize the logger and add log messages as shown below
150+
151+
```
152+
const ocilog= require('../ocilogginganalytics');
153+
var log=ocilog.getlogger();
154+
155+
log.debug("Debug message");
156+
log.warn("Warning message");
157+
log.info("Informational message");
158+
log.error("Error message");
159+
```
160+
<br>
161+
<img width="993" alt="image" src="https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/7a8f59e3-27b2-48ba-814c-283db31ab2b0">
162+
<br>
163+
<img width="986" alt="image" src="https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/0459d163-0e61-41a7-b710-a4467858b8f4">
164+
56165

57166

58167
Next, install the required dependencies
@@ -80,3 +189,33 @@ Make sure to run the metric and tracing files alog with your server
80189
node -r ./metrics.js -r ./tracing.js server.js
81190
```
82191

192+
193+
# Correlating Traces & Logs
194+
OpenTelemetry includes TraceId and SpanId in the LogRecords and this allows to directly correlate logs and traces that correspond to the same execution context.
195+
Application traces and spans flows into OCI Application Performance Monitoring and logs into OCI Logging Analytics service, So OCI APM provides easy way to
196+
navigate from traces and spans to logs in logging analytics in one click using drilldown configuration.
197+
198+
Drilldowns are links to other services in OCI or other custom services using customizable URLs including attributes from spans
199+
(e.g. loganalytics/explorer?search=<OciInstanceId> where OciInstanceId is a span attribute).
200+
201+
202+
Steps to create drilldown to link traces and logs
203+
![CorrelateTraces Logs](https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/f05384ac-d4a6-40f7-90ea-e52ff5d5cdbb)
204+
205+
206+
# Creating Custom Dashboards
207+
208+
Once your tracing data and metrics are in OCI, you can use that data to visually represent it in any way you like. Creating a custom dashboard is easy, just add
209+
the widgets you want to show and modify the source data (metric or trace data). Here are a few links to help you get started with that:
210+
211+
* [Creating custom APM dashboards](https://docs.oracle.com/en-us/iaas/application-performance-monitoring/doc/create-custom-dashboard.html).
212+
* [Creating custom Logging Analytics dashboards](https://docs.oracle.com/en-us/iaas/logging-analytics/doc/create-dashboards.html).
213+
* You can also follow the [blog article on implementing Custom Trace Data Widgets](https://blogs.oracle.com/observability/post/apm-custom-dashboards) to create custom widgets from APM's Trace Explorer queries.
214+
215+
There is an example dashboard which can be found in the /scripts/CustomDashboardExample.json and can be imported into your APM Dashboards.
216+
217+
* Goto APM --> Dashboards --> Import Dashboards --> Select the CustomDashboardExample.json file
218+
219+
![Dashboardimport](https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/061a590a-c63c-4bb8-aa9c-749bd3296c8e)
220+
221+

‎oci-opentelemetry-solution/example/README.md‎

Lines changed: 99 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Product Inventory Application
1616
3. Application launches on - http://localhost:3001
1717

1818

19-
### Enable APM and OpenTelemetry 

19+
### Enable APM and OpenTelemetry
2020
1. Go to the server directory
2121
```
2222
npm install opentelemetry
@@ -33,10 +33,102 @@ Product Inventory Application
3333
7. Edit index.html to include apm-rum.js file
3434

3535
```
36-
<script src="./apm-rum.js"></script>

37-
<script async crossorigin="anonymous" src="[APM DOMAIN ENDPOINT]/static/jslib/apmrum.min.js"></script>



36+
<script src="./apm-rum.js"></script>
37+
<script async crossorigin="anonymous" src="[APM DOMAIN ENDPOINT]/static/jslib/apmrum.min.js"></script>
3838
```
3939

40+
### Enable Logging Analytics
41+
1. Go to the server directory
42+
```
43+
npm install undici
44+
npm install bunyan
45+
npm install oci-loganalytics
46+
47+
```
48+
<br>
49+
50+
2. Copy the ocilogginganalytics.js file to the server directory and edit the file to update the parameters mentioned below.
51+
> Note: Detailed Logging Analytics configuration can be found in oci-observability-and-management/oci-opentelemetry-solution/
52+
53+
[PATH]/config - Path to config file
54+
[PROFILE] - Profile in config file to be used for OCI authentication
55+
[NAMESPACE] - Namespace
56+
[UPLOADNAME] - User defined name for uploads
57+
[LOGSOURCENAME] - Log Source Name created in OCI logging analytics
58+
[LOGGROUPID] - Log Group ID created in Logging analytics to group the log messages
59+
[BUFFERLENGTH] - Buffer size (number of log messages to store)
60+
[FLUSHINTERVAL] – Flush internal in milliseconds to flush messages from buffer and send it to OCI LA
61+
[LOGGERNAME] – user defined name to initialize the bunyan logger
62+
63+
3. Modify the application source files to include log messages as shown below
64+
65+
```
66+
const ocilog= require('../ocilogginganalytics');
67+
var log=ocilog.getlogger();
68+
69+
log.debug("Debug message");
70+
log.warn("Warning message");
71+
log.info("Informational message");
72+
log.error("Error message");
73+
```
74+
75+
Lets modify the file server/routes/items.js to initialize the logger and include log messages as shown below.
76+
77+
```
78+
// Copyright (c) 2023 Oracle and/or its affiliates.
79+
// Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl.
80+
81+
const express = require('express');
82+
const router = express.Router();
83+
const Item = require('../models/Item');
84+
const ocilog= require('../ocilogginganalytics');
85+
var log=ocilog.getlogger();
86+
87+
// Define CRUD routes here (GET, POST, PUT, DELETE)
88+
// Example routes:
89+
90+
// Get all items
91+
router.get('/', (req, res) => {
92+
log.debug("Inside get records method");
93+
Item.find()
94+
.then((items) => res.json(items))
95+
.catch((err) => res.status(400).json('Error: ' + err));
96+
});
97+
98+
// Add a new item
99+
router.post('/', (req, res) => {
100+
log.debug("Inside add record method");
101+
const newItem = new Item({
102+
name: req.body.name,
103+
description: req.body.description,
104+
});
105+
106+
newItem
107+
.save()
108+
.then(() => res.json('Item added!'))
109+
.catch((err) => res.status(400).json('Error: ' + err));
110+
});
111+
112+
// Update an item
113+
router.put('/:id', (req, res) => {
114+
log.debug("Inside update record method");
115+
Item.findByIdAndUpdate(req.params.id, req.body, { new: true })
116+
.then(() => res.json('Item updated!'))
117+
.catch((err) => res.status(400).json('Error: ' + err));
118+
});
119+
120+
// Delete an item
121+
router.delete('/:id', (req, res) => {
122+
log.debug("Inside delete record method");
123+
Item.findByIdAndDelete(req.params.id)
124+
.then(() => res.json('Item deleted.'))
125+
.catch((err) => res.status(400).json('Error: ' + err));
126+
});
127+
128+
module.exports = router;
129+
130+
```
131+
40132
Now lets start the application:
41133

42134
<ins>Terminal 1:</ins>
@@ -51,11 +143,14 @@ cd client
51143
npm start
52144
```
53145

54-
Application traces and metrics will now be sent to OCI APM.
146+
Application traces and metrics will now be sent to OCI APM and application logs will now be sent to OCI Logging Analytics.
55147

56148
### OCI APM - Traces and Spans
57149

58150
![Image-APM](https://github.com/zkhader/oci-observability-and-management/assets/14898804/7ff956f9-d668-4dd2-8cac-79e29562f96f)
59151

152+
### OCI Logging Analytics - Logs
153+
154+
![image](https://github.com/Anand-GitH/oci-observability-and-management/assets/60418080/79043ecf-4103-48d6-b138-3508b00c9491)
60155

61156

0 commit comments

Comments
 (0)