Nsryjdtyk

I have a DynamoDB table with a TTL set on one of the attributes. However when I started ingesting data into the table I accidentally was entering the attribute as a string instead of an integer. This is ignored by the TTL and my records were never being deleted.

After a while I noticed my error and updated the code and the table ceased growing in size as records were being removed as designed. However I still have a few hundred thousand records in the table with the string TTL.

What is the best way for me to delete these records?

I've written a script in python with boto3 which scans the whole table, gets the keys for items with string TTL attributes and deletes them. However deleting the ~500k items seems to be rather slow. So any other answers would be much appreciated.

import boto3

dynamodb = boto3.resource('dynamodb')



table = dynamodb.Table('my-table')



response = table.scan()

data = response['Items']

while 'LastEvaluatedKey' in response:

    response = table.scan(ExclusiveStartKey=response['LastEvaluatedKey'])

    data.extend(response['Items'])



string_keys = [item['key'] for item in data if type(item['ttl']) == str]



with table.batch_writer() as batch:

    for key in string_keys:

        batch.delete_item(

            Key={

                'key': key

            },

    )

    string_keys.remove(key)