lm-logs-aws-integration

This integration provides a CloudFormation template to forward logs from AWS CloudWatch to LogicMonitor.

This CloudFormation template only deploys a log forwarder (lambda function) subscribed to a specific CloudWatch logs group for LogicMonitor. Forwarding logs from individual AWS services, such as EC2, S3, or ELB, should be configured separately.

You will need to supply the following LogicMonitor credentials when configuring the CloudFormation stack:

• LM Access ID
• LM Access Key
• LM Account Name
• LM Account Domain

NOTE: If the LM Account Domain is not specified, it defaults to "logicmonitor.com".

Deploying lambda using CloudFormation

Deploying lambda using Terraform

Sample configuration

variable "lm_access_id" {
  description = "LogicMonitor Access Id"
}

variable "lm_access_key" {
  description = "LogicMonitor Access Key"
}

variable "lm_company_name" {
  description = "LogicMonitor Account Name"
}

variable "lm_company_domain" {
  description = "LogicMonitor Account Domain"
}

# LogicMonitor Logs forwarder
resource "aws_cloudformation_stack" "lm_forwarder" {
  name         = "lm-forwarder"
  capabilities = ["CAPABILITY_IAM", "CAPABILITY_NAMED_IAM", "CAPABILITY_AUTO_EXPAND"]
  parameters   = {
    FunctionName              = "LMLogsForwarder"
    LMAccessId                = var.lm_access_id
    LMAccessKey               = var.lm_access_key
    LMCompanyName             = var.lm_company_name
    LMCompanyDomain           = var.lm_company_domain
    LMRegexScrub              = ""
    PermissionsBoundaryArn    = ""
  }
  template_url = "https://logicmonitor-logs-forwarder.s3.us-west-1.amazonaws.com/source/latest.yaml"
}

terraform apply --var 'lm_access_id=<lm_access_id>' --var 'lm_access_key=<lm_access_key>' --var 'lm_company_name=<lm_company_name>' --var 'lm_company_domain=<lm_company_domain>'

Forwarding EC2 Instances logs

Forward EC2 logs to CloudWatch, using the CloudWatch Logs Agent. Note: The logstream name typically defaults to the instance ID (this is expected by LogicMonitor). After you have started recieving your EC2 logs in the desired log group:

1. Go to CloudWatch, select the desired log group of which you want to forward logs , under Actions > Create Lambda subscription filter
2. In Create Lambda subscription filter , select "Lambda Function" and choose "LMLogsForwarder" (or, whatever you named the Lambda function during stack creation) and click Start streaming.

Forwarding S3 bucket access logs

To forward S3 bucket access logs to LogicMonitor:

1. Under the bucket Properties, enable Server access logging. You will need to select a Target bucket where the access logs will be stored. If this target bucket doesn't exist, you need to create it. (This is different from the source bucket.)
2. Go to the target bucket, and under Advanced settings > Events add a notification for "All object create events".
3. Send to "Lambda Function" and choose "LMLogsForwarder" (or, whatever you named the Lambda function during stack creation).

Forwarding ELB access logs

To send ELB access logs to LogicMonitor:

1. In the EC2 navigation pane, choose Load Balancers and select your load balancer.
2. Under Attributes > Access logs, click "Configure access logs".
3. Select "Enable access logs" and specify the S3 bucket to store the logs. (You can create one, if it doesn't exist.)
4. Go to the S3 bucket (from Step 3), and under Advanced settings > Events add a notification for "All object create events".
5. Send to "Lambda Function" and choose "LMLogsForwarder" (or, whatever you named the Lambda function during stack creation).

Forwarding RDS logs

To send RDS logs to LogicMonitor, configure instance to send logs to cloudwatch, and create subscription filter to send logs to the LM log forwarder:

1. Follow instructions to send standard RDS logs to cloudwatch or enhanced RDS to cloudwatch
2. Go to Cloudwatch, select the desired log group of which you want to forward logs , under Actions > Create Lambda subscription filter
3. In Create Lambda subscription filter , select "Lambda Function" and choose "LMLogsForwarder" (or, whatever you named the Lambda function during stack creation) and click Start streaming.

Forwarding Lambda logs

To send Lambda logs to LogicMonitor, go to cloudwatch and find lambda's log group, and create subscription filter to send logs to the LM log forwarder:

1. Go to Cloudwatch, select the lambda's log group of which you want to forward logs , under Actions > Create Lambda subscription filter
2. In Create Lambda subscription filter , select "Lambda Function" and choose "LMLogsForwarder" (or, whatever you named the Lambda function during stack creation) and click Start streaming.

Forwarding EKS logs

Add an "Amazon CloudWatch Observability" plugin to existing or new cluster, OR Forward EKS logs to cloudwatch using application metrics to cloudwatch and application logs to cloudwatch using Fluenbit

These steps create 5 different log groups into cloudwatch as below:

1. /aws/containerInsights//application
2. /aws/containerInsights//host
3. /aws/containerInsights//performance
4. /aws/containerInsights//dataplane
5. /aws/eks//cluster

To forward EKS logs to LogicMonitor, follow these steps for the specific log group you want to send:

1. Go to Cloudwatch, select the EKS's log group of which you want to forward logs , under Actions > Create Lambda subscription filter
2. In Create Lambda subscription filter , select "Lambda Function" and choose "LMLogsForwarder" (or, whatever you named the Lambda function during stack creation) and click Start streaming.

Send flow logs from EC2

1. Add below lines in permissions of lambda's role policy: "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"
2. Add below line in the Trust Relationship part of the role in the Service tag: "vpc-flow-logs.amazonaws.com"
3. A Log group in cloud watch should be created with name /aws/ec2/networkInterface
4. Use the instance id of your EC2 instance to search in Network interfaces page. Select that Network interface row and create a flow log. In create flow log Destination log group should be /aws/ec2/networkInterface and IAM role should be the role created in 1st and 2nd step.
5. In Log record format, select Custom Format. Log format should have first value as instance-id. Rest of the values can be as per your requirements. For details on different fields please refer to Available fields section on https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html.
6. Go to /aws/ec2/networkInterface log group. In Actions > Subscription filters > Create lambda subscription filter. In lambda function select “LMLogsForwarder” (or whatever you named the Lambda function during stack creation) and provide Subscription filter name. Hit Start Streaming.

Send flow logs from NAT Gateway

1. Add below lines in permissions of lambda's role policy: "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"
2. Add below line in the Trust Relationship part of the role in the Service tag: "vpc-flow-logs.amazonaws.com"
3. A Log group in cloud watch should be created with name /aws/natGateway/networkInterface
4. Use the your Nat Gateway id to search in Network interfaces page. Select that Network interface row and create a flow log. In create flow log Destination log group should be /aws/natGateway/networkInterface and IAM role should be the role created in 1st and 2nd step.
5. Go to /aws/natGateway/networkInterface log group. In Actions > Subscription filters > Create lambda subscription filter. In lambda function select “LMLogsForwarder” (or whatever you named the Lambda function during stack creation) and provide Subscription filter name. Hit Start Streaming.

Send logs from Cloudtrail

1. On CloudTrail page click Create Trail button.
2. Provide Trail name. Unselect checkbox naming "Log file SSE-KMS encryption" if you do not want to SSE-KMS encrypt your log files.
3. Make sure to select CloudWatch Logs Enabled checkbox and provide log group name as "/aws/cloudtrail".
4. If you have existing IAM role Cloudtrail permissions, provide it as input in IAM role box. Else a new role can also be created, make sure to provide a name for the new role.
5. In the next page choose the type of logs that you would like to be collected.
6. In the next page review the provided configuration and hit Create trail button.
7. Go to Cloudwatch's Log group page and go in /aws/cloudtrail log group.
8. In Actions > Subscription filters > Create lambda subscription filter. In lambda function select “LMLogsForwarder” (or whatever you named the Lambda function during stack creation) and provide Subscription filter name. Hit Start Streaming.
9. Logs will start to propagate through lambda to LogIngest. You will be able to see logs against AWS account name resource.

Send logs from Cloudfront

1. In Cloudfront page Select the distribution for which you would like to collect logs.
2. In Standard Logging Select "On" radio button.
3. In S3 bucket for logs Select the bucket in which you would want to store the logs.
4. Click Create Distribution.
5. Go to S3 bucket that you had selected in 3rd step.
6. Go to Properties page. Select Create event notification button in Event notifications tab.
7. Provide Event name. In Destination's Lambda function tab select “LMLogsForwarder” (or whatever you named the Lambda function during stack creation).
8. Click Save changes button.
9. You will be able to see logs at logicmonitor website against S3 bucket mentioned in 3rd step.

Send Logs from Kinesis Data Stream:

As these logs are filtered from Cloudtrail, all the Cloudtrail steps needs to be implemented. No separate process is needed for Kinesis Data Stream.

Send Logs from Kinesis Firehose:

There are 2 kinds of logs in Kinesis Firehose API logs which will be collected from Cloudtrail and second are error logs.

For API logs you don't have to do anything extra other than Cloudtrail steps.

For Error logging:

1. While Creating delivery system in Configure System step, select Enabled radio button in Error logging segment.
2. A log group in Cloudwatch would be created with delivery system's name. Format of name would be /aws/kinesisfirehose/.
3. In Actions > Subscription filters > Create lambda subscription filter. In lambda function select “LMLogsForwarder” (or whatever you named the Lambda function during stack creation) and provide Subscription filter name. Hit Start Streaming.
4. Logs will start to propagate through lambda to LogIngest. You will be able to see logs against Kinesis Firehose delivery system's name.

Send Logs from ECS:

As these logs are filtered from Cloudtrail, all the Cloudtrail steps needs to be implemented. No separate process is needed for ECS.

Send ELB flow logs

ELB flow logs to ECS:

Send RDS logs

1.Add below lines in permissions of lambda's role policy: "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"

2. Add below line in the Trust Relationship part of the role in the Service tag: "vpc-flow-logs.amazonaws.com"
3. A Log group in cloud watch should be created with name /aws/rds/networkInterface
4. Use your RDS instance private IP address to search in Network interfaces page. Select that Network interface row and create a flow log. In create flow log Destination log group should be /aws/rds/networkInterface and IAM role should be the role created in 1st and 2nd step.
5. Go to /aws/rds/networkInterface log group. In Actions > Subscription filters > Create lambda subscription filter. In lambda function select “LMLogsForwarder” (or whatever you named the Lambda function during stack creation) and provide Subscription filter name. Hit Start Streaming.
6. Logs will start to propagate through lambda to LogIngest.

Send Bedrock logs

1. There are two types of logs supported by AWS Bedrock that can be sent to AWS Cloudwatch: Model invocation logging and Knowledge Base Logging
2. For setting up the Model Invocation Logging follow [model invocation logging] (https://docs.aws.amazon.com/bedrock/latest/userguide/model-invocation-logging.html). For sending logs from the Knowledge base to Cloudwatch follow [Knowledge base logging] (https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-bases-logging.html)
3. A Log group in cloud watch should be created with name that contains "bedrock" in it.
4. To differentiate between modelInvocation logs and knowledge-base logs, for knowledge-base logs the Log group name should contain "knowledge-base" or "vendedlogs" (this is by default present so no need to modify it). For modelinvocation logs, the logstreams by default contain string "modelinvocations" in it.
5. Go to Log Group created by bedrock as above. In Actions > Subscription filters > Create lambda subscription filter. In lambda function select “LMLogsForwarder” (or whatever you named the Lambda function during stack creation) and provide Subscription filter name. Hit Start Streaming.
6. Logs will start to propagate through lambda to LogIngest.
7. The Model Invocation logs will be mapped to the Bedrock model resource created in Logicmonitor and the knowledge-base logs will be mapped to the AWS account resource created in Logicmonitor.

Send Q Business Logs

1. Types of Logs Supported by AWS Q Business:

   • User Conversation Logs:
     • Content: Records of user interactions, including messages and responses.
     • Purpose: Analyze user behavior, identify common queries, and assess response quality.
   • Feedback Logs:
     • Content: User feedback on responses, such as thumbs-up or thumbs-down ratings.
     • Purpose: Gauge user satisfaction and identify areas for improvement.
   • API Call Logs:
     • Content: Details of API requests made to Amazon Q Business services.
     • Purpose: Audit API usage, monitor access patterns, and troubleshoot issues.

2. Steps to Enable Log Streaming to Amazon CloudWatch:

   • Amazon CloudWatch Logs Setup:
     • In the Amazon Q Business console, navigate to your application environment.
     • Go to Enhancements > Admin Controls and Guardrails.
     • Under Log delivery, choose Add and select Amazon CloudWatch Logs.
     • Specify the destination log group , make sure to add "/aws/vendedlogs/qbusiness/" to the Destination Log Group and configure additional settings as needed.
     • Reference: Enabling Amazon Q Business user conversation logging

3. Sending logs from Cloudwatch to LM Portal

   • In the Log Group created for QBusiness logs, go to Subscription Filter -> Create -> Create Lambda Subscription Filter
   • In lambda function select “LMLogsForwarder” (or whatever you named the Lambda function during stack creation) and provide Subscription filter name.
   • Specify Subscription filter name, and click start streaming.

Send Sagemaker logs

To send AWS SageMaker logs to AWS CloudWatch, three log types are supported: Training Job Logs, Processing Job Logs, and Endpoint Logs. For each log type, you need to create the corresponding instances, which will generate CloudWatch log groups where the logs will be sent. These log groups include:

aws/Sagemaker/TrainingJobs aws/Sagemaker/ProcessingJobs aws/Sagemaker/<endpoint_name>

To set up log forwarding, follow these steps:

1. Navigate to the log group created by SageMaker in CloudWatch.
2. Go to Actions > Subscription Filters > Create Lambda Subscription Filter.
3. In the Lambda function dropdown, select the function you created (e.g., “LMLogsForwarder”).
4. Provide a name for the subscription filter and click Start Streaming.

The logs will now flow through the Lambda function to LogIngest. SageMaker endpoint logs will be mapped to the corresponding endpoint resource in LogicMonitor, while the Training and Processing Job logs will be mapped to the AWS account resource created in LogicMonitor.

Send Q Business API Call Logs using cloudtrail

To send API call logs from AWS Q Business to LogicMonitor using cloudtrail, follow these steps:

1. Enable CloudTrail for Management Events:
   • Open the AWS Console > CloudTrail > Trails
   • Click Create trail or edit an existing trail
   • Under Management events:
     • Enable Read and Write management events
     • This will capture all AwsApiCall events — including those from qbusiness.amazonaws.com
2. Forward Events to CloudWatch Logs
   • In the same CloudTrail trail configuration, under CloudWatch Logs:
     • Enable CloudWatch Logs
     • Specify a log group name (e.g., /aws/cloudtrail/qbusiness)
     • Create or select an IAM role that allows CloudTrail to publish logs to CloudWatch
3. Create Lambda Subscription Filter
   • Go to the CloudWatch Logs console
   • Select the log group created in step 2 (e.g., /aws/cloudtrail/qbusiness)
   • Click on Actions > Create Lambda subscription filter
   • Choose the Lambda function you created (e.g., “LMLogsForwarder”)
   • Provide a name for the subscription filter and click Start streaming
4. Verify Logs in LogicMonitor
   • After a few minutes, check your LogicMonitor portal to see if the API call logs from AWS Q Business are being ingested correctly.
   • The logs should be associated with the AWS account resource created in your LogicMonitor portal.