AWS Lambda invoke on existing S3 objects in python

Posted on January 11, 2018 in misc

In This article explain how to invoke lambda on existing s3 object.

As we know lambda can be invoke on Object event like creation etc. There is no way to handle existing object or there is no such event which will invoke lambda on existing s3 object.

All we have to do to invoke lambda manually for each and every s3 object.

I achieved this by iterate through each file in your target S3 bucket and for each it will execute the desired lambda function against it emulating a put operation.

You're probably going to want to put a very long execution time allowance against this function

Steps:

create record for every s3 object as given below: (append every s3 record into Records list)

 params = {
            'Records': [
                {
                    'eventVersion': '2.0',
                    'eventTime': u'2018-01-16T07:54:28.323Z',
                    'requestParameters': {
                        'sourceIPAddress': u'123.201.192.96'
                        },
                    's3': {
                        'configurationId': u'task-dev:function.method_name',
                        'object': {
                            'eTag': 'a961fdbd7cc2a2522b3c453181965573',
                            'sequencer': '005A5DAFB445353DF7',
                            'key': 'Screen+Shot+2017-11-27+at+11.57.40+AM.png',
                            'size': 487383
                        },
                        'bucket': {
                            'arn': 'arn:aws:s3:::sample_bucket_name',
                            'name': 'sample_bucket_name',
                            'ownerIdentity': {'principalId': u'AP0LPLI499QA'}
                        },
                        's3SchemaVersion': '1.0'
                    },
                    'responseElements': {
                        'x-amz-id-2': '8DgYnJE6zF2TVw0V4ES9BCM1GeXMa0xFDlivb2+HvPpmh7paFfg5K5xwfxQVEmFfH476O1uPQHw=',
                        'x-amz-request-id': u'157B0FB8B32D4A40'
                    },
                    'awsRegion': u'us-west-2',
                    'eventName': u'ObjectCreated:Put',
                    'userIdentity': {
                        'principalId': 'AWS:AIDAI2FTBVZZPQ3OACVBU'
                    },
                    'eventSource': 'aws:s3'}
            ]
        }

just make sure in above params "s3.object", "bucket" are proper.
next step is to install boto3 lib using pip (pip install boto3).
make sure you have properly configure aws credential in your system.
lambda_client = boto3.client('lambda', region_name='us-west-2')
now invoke lambda
response = lambda_client.invoke(FunctionName="", InvocationType = 'Event', Payload = simplejson.dumps(params))
response will return 202 as status code with other informations. and your done !!!!