Core data migration: Set a unique constraint to a parameter and avoid duplicates

Coredata migrations are easier said than done, isn’t it? One wrong step and we are doomed. 😥 

Coming from an Android background (where the platform allows writing raw SQLite queries in the migration), it feels tough to handle manual migration on iOS. Although I love the fact that lightweight migration works out of the box, when it comes to other types of migrations, it’s just too many tasks to remember, and it feels like I’m defusing a bomb.


We have recently seen duplicate records saved in one of our entities. Our investigation made it clear that the uniqueId parameter (column) we have in the entity isn’t unique and is an optional parameter. 

Though we had implemented a code logic that handles upsert operation (update/insert the record based on the record availability in the entity), it was not sufficient. The duplicate record will still be saved if there’s a race condition. So, we decided to solve this at the database level. 

When I started working on it, I referred to a lot of documentation & resources but couldn’t get complete information. Most of the articles only talked about lightweight migration. Now that we have successfully migrated, I am sharing my learnings here and hope that it will help somebody who is dealing with a similar problem. 

Let’s jump right into it.

Existing Set-up

For better understanding, I’ve created a sample project with a single entity, UserEntity, in its data model. 

C02O3EFXzc8FBVQfc74a0J69b9Ojdz8J1ttdSOHJqEY8HpnJ4VUV cwfMNx2HX3EjDNGmbnym1O PSieya95yAV xXj79PF1hX2KvY jaj4bHRnbRf ZPp4wM07dr4e4LX

UserEntity with its parameters. Observe UniqueId is an optional parameter here.

CoreData creates a table in SQLite under the hood when we run the project. The generated table structure can be seen below. A minor detail to observe here is the naming convention — table name & parameter names starts with Z, and PrimaryKey Z_PK is internally created and handled. 

iM7guREZnjQToL603OBCfoA6gzrfCcy9UayFUW W7ASUrSu6jEm1zw9yOnQ3jUFBODA bfSZQ0J0DjRHn4whMKDrPZgPipAqs0ht76bOD1MSMMVYQDk24psqMvhrscP3EFSIpuwSkkNCSIAhddSqPkTTpWlf sKgTTKnMBIR p43vtAphhFa2yL3dg

UserEntity table created in SQLite under the hood.


With this setup, every time we insert a record, CoreData treats it as a new record. Hence it can store duplicate records (even when uniqueId is the same) as no unique constraint has been set. 

If the app is still under development, we could make the necessary changes on UserEntity to not accept duplicate records. What if this implementation has already been rolled-out to production, users already using the app, and they see duplicate records? Now, only a migration can fix the issue.

Coredata migration process

  1. Create a second version of the data model and set it as a current model. This creates a copy of the existing data model.

2. Add uniqueId parameter in UserEntity’s Constraints section in the data model v2. This adds a unique constraint to the parameter.

HqzBe9xaULXF5fvhq6nU Jeoh e6PL9roGSx2FC 5uGuCzDAQ8M hpAT4yeAkGCWPL8uFB8LkO5U 0c4ZkG1i 8B3tbFtJfOjZbB6cZIvLoEYM WDJB deGRbhhxI0w028H

3. Set NSMergePolicy to handle the merge conflict while inserting the record. Use context.mergePolicy = NSMergePolicy.mergeByPropertyObjectTrump. This MergePolicy asks Core Data to merge duplicate objects based on their properties. So, if a record with the same uniqueId is already present, CoreData will update the existing record instead of inserting a new one.

Afne FXuACBK01Cx3oioefIbPm wt4VQL4eTkoJurLvmS428dAYKbNsLbF7sb3N9 R3i ztjc1FHjrrFS4eDjDSrX2dIjZ7nZQs16qgfHckP9 Xrsq09XO6SWAzk530dkryTjl4e2ONfiiynTlgp4BUk 2dgKTcf k9BOJwCde5SoQgAPeXTGZ2X5g

A sample implementation of NSMergePolicy

When we run the app, the migration would fail as lightweight migration cannot handle the unique constraint migration. This needs a manual migration with the help of the mapping model.

NSUnderlyingException=Constraint unique violation: UNIQUE constraint failed: ZUSERENTITY.ZUNIQUEID, reason=constraint violation during attempted migration, NSExceptionOmitCallstacks=true}}}, [“reason”: Cannot migrate store in-place: constraint violation during attempted migration, “NSUnderlyingError”: Error Domain=NSCocoaErrorDomain Code=134111 “(null)”

4. Let’s create a mapping model by selecting v1 as the source data model and v2 as the target data model. gif maker

Creating a mapping model is insufficient because it only maps the old records to the new data model. If we run the app, it would still crash by showing the same error. So, how do we solve the unique constraint exception? 

The answer is to use NSEntityMigrationPolicy

The app crashes when we run the app because CoreData wouldn’t know how to handle the existing duplicate records (if any). Since we have set the unique constraint to uniqueId, we should remove the existing duplicate records from the source data model before the migration takes place.

5. So, let’s create a custom MigrationPolicy that deletes the duplicate records. 

kmyD4WwzlEEOu0EJP9x7YVDvRFaE7nJv3wxIj3zh7sI0uYaUXvy6j1Kd7LyDdRajfV6FhuHVMJXP5OL9vaUR4HAYFKZtXO fiduFbIeE1uJ6amP9YqoYxZQoDmgU3llVk1 5OmydHgIQs3KXTgflE0INQlyv1bIx iB0916tgSD1f3ELL4LzhHBOVw

Sample code snippet to remove duplicate records from the source context

NSEntityMigrationPolicy has a begin(mapping:manager) function that the migration manager invokes at the start of the given entity mapping. We are making use of this function to run our code snippet. This code snippet gets all the records from sourceContext’s UserEntity and deletes duplicate records if any. 

6. The final step is to tell the CoreData to use this MigrationPolicy when migrating the UserEntity table. We can do so by specifying the fully name-spaced class name of MigrationPolicy_v1_v2 in the MappingModel. Make sure the entity mappingType changes from copy to custom once you add the custom policy.

K1QQHy4BdQzngO6QqOTCQqbLMRe39TeQ60PQDEw7PXFIJ0BgBXjW021NbO1 26X2bhztOKwQbhFTuWqDdzuJA

Final words

Were we able to run the migration successfully? Oh yeah! 

If we do all these steps correctly and run the app, coredata migration succeeds and removes the existing duplicate records as well. Any future insertions will avoid duplicates by updating the existing record if it is already available in the table.

I have created a sample app to implement & test the functionality. You can refer to the source code for a detailed understanding.

Recommended Reads