How To Synchronize Core Data with a Web Service – Part 2

Learn how to synchronize Core Data with a web service!

This is a post by iOS Tutorial Team Member Chris Wagner, an enthusiast in software engineering always trying to stay ahead of the curve. You can also find him on Google+.

Welcome back to our 2-part tutorial series on how to synchronize core data with a web service!

Just to refresh your memory, here’s what you did in the first part of this series:

1. Downloaded and ran the starter project
2. Setup a free Parse account
3. Wrote an AFNetworking client to talk to the Parse REST API
4. Created a “Sync Engine” Singleton class to handle synchronization
5. Processed web service data into Core Data
6. Manually triggered sync with remote service

The net result of all that hard work above was that you ended up with an App that tracks important dates, and synchronizes that data with the online storage service. While that’s incredibly cool, you can make that App even more awesome by completing this second and final part of the series!

Here you will complete three more vital pieces to round out your App:

1. Delete local objects when deleted on server
2. Push records created locally to remote service
3. Delete records on server when deleted locally

If you did not complete part 1, lost your project, or just want to start the tutorial knowing your code is in sync, don’t sweat it! :] You can download everything covered in Part 1 here.

If you do choose to use this file, make sure to replace values for kSDFParseAPIApplicationId and kSDFParseAPIKey with the values provided to you from the Overview tab of the Parse project window. Also, make sure to build and run the program before going any further just to make sure that everything is in working order.

Ready? Let’s dive in to deletion!

Delete local objects when deleted on server

To be sure that local objects are deleted when they no longer exist on the server, your app will download all of the records on the server and compare them with what you have locally. It’s assumed that any record you have locally, that does not exist on the server, should be deleted.

One untoward side effect with the Parse REST API is that this approach causes some overhead as it retrieves the full objects, instead of just the objectID fields. (Holy data usage, Batman!) Alternatively, you could have a delete flag on your records on the remote service and retrieve all records matching with the deleted flag set. While this approach would reduce overhead, the downside is that you can never actually delete records from the server — the records will stick around in perpetuity.

First, in SDSyncEngine.m, update the signature of:

- (void)downloadDataForRegisteredObjects:(BOOL)useUpdatedAtDate {

To be:

- (void)downloadDataForRegisteredObjects:(BOOL)useUpdatedAtDate toDeleteLocalRecords:(BOOL)toDelete {

And then update your -startSync method to reflect the new signature:

- (void)startSync {
    if (!self.syncInProgress) {
        [self willChangeValueForKey:@"syncInProgress"];
        _syncInProgress = YES;
        [self didChangeValueForKey:@"syncInProgress"];
        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
            [self downloadDataForRegisteredObjects:YES toDeleteLocalRecords:NO];
        });
    }
}

Now you need a method to process the deletion of these local records. Add the following method below -processJSONDataRecordsIntoCoreData:

- (void)processJSONDataRecordsForDeletion {
    NSManagedObjectContext *managedObjectContext = [[SDCoreDataController sharedInstance] backgroundManagedObjectContext];
    //
    // Iterate over all registered classes to sync
    //
    for (NSString *className in self.registeredClassesToSync) {
        //
        // Retrieve the JSON response records from disk
        //
        NSArray *JSONRecords = [self JSONDataRecordsForClass:className sortedByKey:@"objectId"];
        if ([JSONRecords count] > 0) {
            //
            // If there are any records fetch all locally stored records that are NOT in the list of downloaded records
            //
            NSArray *storedRecords = [self 
                                      managedObjectsForClass:className 
                                      sortedByKey:@"objectId" 
                                      usingArrayOfIds:[JSONRecords valueForKey:@"objectId"] 
                                      inArrayOfIds:NO];
            
            //
            // Schedule the NSManagedObject for deletion and save the context
            //
            [managedObjectContext performBlockAndWait:^{
                for (NSManagedObject *managedObject in storedRecords) {
                    [managedObjectContext deleteObject:managedObject];
                }
                NSError *error = nil;
                BOOL saved = [managedObjectContext save:&error];
                if (!saved) {
                    NSLog(@"Unable to save context after deleting records for class %@ because %@", className, error);
                }
            }];
        }
        
        //
        // Delete all JSON Record response files to clean up after yourself
        //
        [self deleteJSONDataRecordsForClassWithName:className];
    }
    
    //
    // Execute the sync completion operations as this is now the final step of the sync process
    //
    [self executeSyncCompletedOperations];
}

Next, update the implementation of -downloadDataForRegisteredObjects:toDeleteLocalRecords: to take into consideration this new BOOL, toDelete.:

    ...
    
    [[SDAFParseAPIClient sharedClient] enqueueBatchOfHTTPRequestOperations:operations progressBlock:^(NSUInteger numberOfCompletedOperations, NSUInteger totalNumberOfOperations) {
        
    } completionBlock:^(NSArray *operations) {
        if (!toDelete) {
            [self processJSONDataRecordsIntoCoreData];
        } else {
            [self processJSONDataRecordsForDeletion];
        }
    }];
    
    ...

The only changes are in completionBlock for enqueueBatchOfHTTPRequestOperations..

Now that -processJSONDataRecordsIntoCoreData is no longer the last method to be executed in the sync process, you must remove the following line from its implementation:

[self executeSyncCompletedOperations];

The final lines of this method should now look like:

...
        [managedObjectContext performBlockAndWait:^{
            NSError *error = nil;
            if (![managedObjectContext save:&error]) {
                NSLog(@"Unable to save context for class %@", className);
            }
        }];
        
        [self deleteJSONDataRecordsForClassWithName:className];
    }
    [self downloadDataForRegisteredObjects:NO toDeleteLocalRecords:YES];
}

Now build and run the App! Once the App is running, go to the Parse Data Browser and delete one of your records. After deleting the record go back to the App and press the Refresh button to see it disappear! Holy cow — like magic, it’s gone! Now you can add and remove records from the remote service and your App will always stay in sync.

Push records created locally to remote service

In this section, you will create a feature that will push records created within the App to the remote service. Start by adding a new method to SDAFParseAPIClient to handle this communication. Open SDAFParseAPIClient.h and add the following method declaration:

- (NSMutableURLRequest *)POSTRequestForClass:(NSString *)className parameters:(NSDictionary *)parameters;

Now implement this method:

- (NSMutableURLRequest *)POSTRequestForClass:(NSString *)className parameters:(NSDictionary *)parameters {
    NSMutableURLRequest *request = nil;
    request = [self requestWithMethod:@"POST" path:[NSString stringWithFormat:@"classes/%@", className] parameters:parameters];
    return request;
}

This new method simply takes a className and an NSDictionary of parameters which is your JSON data to post to the web service. Take a look at creating objects with the Parse REST API for some background on what you’ll be doing in this next step. Essentially, it is a standard HTTP POST — so if you have some web experience with HTTP operations, this should be very familiar!

Next you need a way to determine which records are to be pushed to the remote service. You already have an existing syncStatus flag in SDSyncEngine.h which can be used for this purpose. Update your enum to look like:

typedef enum {
    SDObjectSynced = 0,
    SDObjectCreated,
} SDObjectSyncStatus;

You will then need to use this new flag when objects are created locally. Go to SDAddDateViewController.m and import the SDSyncEngine header so that the enum is visible:

#import "SDSyncEngine.h"

Next, update the -saveButtonTouched: method to set the syncStatus flag to SDObjectCreated when a new record is added:

- (IBAction)saveButtonTouched:(id)sender {
    if (![self.nameTextField.text isEqualToString:@""] && self.datePicker.date) {
        [self.date setValue:self.nameTextField.text forKey:@"name"];
        [self.date setValue:[self dateSetToMidnightUsingDate:self.datePicker.date] forKey:@"date"];
        // Set syncStatus flag to SDObjectCreated
        [self.date setValue:[NSNumber numberWithInt:SDObjectCreated] forKey:@"syncStatus"];
        if ([self.entityName isEqualToString:@"Holiday"]) {
	    ...

When a new record is added, it will be handy to attempt to push the record to the remote service immediately, in order to save the user a sync step later. Update the addDateCompletionBlock in -prepareForSegue:sender in order to call startSync in the addDateCompletionBlock to immediately push the record to the remote service.

        [addDateViewController setAddDateCompletionBlock:^{
            [self loadRecordsFromCoreData]; 
            [self.tableView reloadData];
            [[SDSyncEngine sharedEngine] startSync];
        }];

In order to send your Core Data records to the remote service you must translate them to the appropriate JSON format for the remote service and use your new method in SDAFParseAPIClient. The JSON string for Holidays and Birthdays will be different, so you will need two different methods. In order to keep the sync engine decoupled from your Core Data entities, add a category method on NSManagedObject which can be called from the sync engine to get a JSON representation of the record in Core Data.

Go to File\New\File…, choose iOS\Cocoa Touch\Objective-C category, and click Next. Enter NSManagedObject for Category on, name the new category JSON, click Next and Create.

You will now have two new files, NSManagedObject+JSON.h and NSManagedObject+JSON.m. Add two new method declarations in NSManagedObject+JSON.h:

- (NSDictionary *)JSONToCreateObjectOnServer;
- (NSString *)dateStringForAPIUsingDate:(NSDate *)date;

This method will return an NSDictionary which represents the JSON value required to create the object on the remote service. You will use NSDictionary as it is easy to create in pure Objective-C syntax and it is what your POST method in SDAFParseAPIClient expects. AFNetworking will take care of the task to convert the NSDictionary to a string for you when sending the POST request to the server.

Implement the category method in NSManagedObject+JSON.m:

- (NSDictionary *)JSONToCreateObjectOnServer {
    @throw [NSException exceptionWithName:@"JSONStringToCreateObjectOnServer Not Overridden" reason:@"Must override JSONStringToCreateObjectOnServer on NSManagedObject class" userInfo:nil];
    return nil;
}

This looks like a rather odd implementation, doesn’t it? The issue here is that there is no generic implementation possible for this method. ALL of the NSManagedObject subclasses must implement this method themselves by overriding it. Whenever a NSmanagedObject subclass does NOT implement this method an exception will be thrown – just to keep you in line! :]

Note: A word of caution – in this next step, you’re about to edit some derived files. If you edit your Core Data model and regenerate these defined files your changes will be lost! It’s highly annoying and time-wasting when you forget to do this, so be careful! One way to get around this problem is to generate a category on the NSManagedObject subclass just as you did for NSManagedObject+JSON; all of your custom methods go in the category and you won’t lose them when you regenerate the file. You know what they say — a line of code in time saves nine…or something like that! :]

Open Holiday.m and import the category and sync engine headers:

#import "NSManagedObject+JSON.h"
#import "SDSyncEngine.h"

Now go ahead and Implement the -JSONToCreateObjectOnServer method:

- (NSDictionary *)JSONToCreateObjectOnServer {
    NSDictionary *date = [NSDictionary dictionaryWithObjectsAndKeys:
                          @"Date", @"__type",
                          [[SDSyncEngine sharedEngine] dateStringForAPIUsingDate:self.date], @"iso" , nil];
    
    NSDictionary *jsonDictionary = [NSDictionary dictionaryWithObjectsAndKeys:
                                    self.name, @"name",
                                    self.details, @"details",
                                    self.wikipediaLink, @"wikipediaLink",
                                    date, @"date", nil];    
    return jsonDictionary;
}

This implementation is fairly straightforward. You’ve built an NSDictionary that represents the JSON structure required by the remote services API. First the code builds the required structure for the Date field, and then builds the rest of the structure and passes in your date NSDictionary.

Now do the same for Birthday.m:

#import "NSManagedObject+JSON.h"
#import "SDSyncEngine.h"

Don’t neglect to Import the category and sync engine headers:

- (NSDictionary *)JSONToCreateObjectOnServer {
    NSDictionary *date = [NSDictionary dictionaryWithObjectsAndKeys:
                          @"Date", @"__type",
                          [[SDSyncEngine sharedEngine] dateStringForAPIUsingDate:self.date], @"iso" , nil];
    
    NSDictionary *jsonDictionary = [NSDictionary dictionaryWithObjectsAndKeys:
                                    self.name, @"name",
                                    self.giftIdeas, @"giftIdeas",
                                    self.facebook, @"facebook",
                                    date, @"date", nil];
    return jsonDictionary;
}

Eagle-eyed readers will note that it’s exactly the same code, with the appropriate properties for Birthday objects instead of Holiday objects.

As noted in part 1, the Parse date format is just a teeny bit different than NSDate — but just enough to cause you a bit of extra work. You’ll need a small function to make the necessary changes to date strings. Add a method and its interface declaration to SDSyncEngine.h:

- (NSString *)dateStringForAPIUsingDate:(NSDate *)date;

And to SDSyncEngine.m:

- (NSString *)dateStringForAPIUsingDate:(NSDate *)date {
    [self initializeDateFormatter];
    NSString *dateString = [self.dateFormatter stringFromDate:date];
    // remove Z
    dateString = [dateString substringWithRange:NSMakeRange(0, [dateString length]-1)];
    // add milliseconds and put Z back on
    dateString = [dateString stringByAppendingFormat:@".000Z"];
    
    return dateString;
}

Now to use the new category in SDSyncEngine.m:

#import "NSManagedObject+JSON.h"

Import your NSManagedObject JSON Category and add the following method, beneath -newManagedObjectWithClassName:forRecord:

- (void)postLocalObjectsToServer {
    NSMutableArray *operations = [NSMutableArray array];
    //
    // Iterate over all register classes to sync
    //    
    for (NSString *className in self.registeredClassesToSync) {
        //
        // Fetch all objects from Core Data whose syncStatus is equal to SDObjectCreated
        //
        NSArray *objectsToCreate = [self managedObjectsForClass:className withSyncStatus:SDObjectCreated];
        //
        // Iterate over all fetched objects who syncStatus is equal to SDObjectCreated
        //
        for (NSManagedObject *objectToCreate in objectsToCreate) {
            //
            // Get the JSON representation of the NSManagedObject
            //
            NSDictionary *jsonString = [objectToCreate JSONToCreateObjectOnServer];
            //
            // Create a request using your POST method with the JSON representation of the NSManagedObject
            //
            NSMutableURLRequest *request = [[SDAFParseAPIClient sharedClient] POSTRequestForClass:className parameters:jsonString];
            
            AFHTTPRequestOperation *operation = [[SDAFParseAPIClient sharedClient] HTTPRequestOperationWithRequest:request success:^(AFHTTPRequestOperation *operation, id responseObject) {
                //
                // Set the completion block for the operation to update the NSManagedObject with the createdDate from the 
                // remote service and objectId, then set the syncStatus to SDObjectSynced so that the sync engine does not
                // attempt to create it again
                //
                NSLog(@"Success creation: %@", responseObject);
                NSDictionary *responseDictionary = responseObject;
                NSDate *createdDate = [self dateUsingStringFromAPI:[responseDictionary valueForKey:@"createdAt"]];
                [objectToCreate setValue:createdDate forKey:@"createdAt"];
                [objectToCreate setValue:[responseDictionary valueForKey:@"objectId"] forKey:@"objectId"];
                [objectToCreate setValue:[NSNumber numberWithInt:SDObjectSynced] forKey:@"syncStatus"];
            } failure:^(AFHTTPRequestOperation *operation, NSError *error) {
                // 
                // Log an error if there was one, proper error handling should be done if necessary, in this case it may not
                // be required to do anything as the object will attempt to sync again next time. There could be a possibility
                // that the data was malformed, fields were missing, extra fields were present etc... so it is a good idea to
                // determine the best error handling approach for your production applications.                
                //
                NSLog(@"Failed creation: %@", error);
            }];
            //
            // Add all operations to the operations NSArray
            //
            [operations addObject:operation];
        }
    }
    
    //
    // Pass off operations array to the sharedClient so that they are all executed
    //
    [[SDAFParseAPIClient sharedClient] enqueueBatchOfHTTPRequestOperations:operations progressBlock:^(NSUInteger numberOfCompletedOperations, NSUInteger totalNumberOfOperations) {
        NSLog(@"Completed %d of %d create operations", numberOfCompletedOperations, totalNumberOfOperations);
    } completionBlock:^(NSArray *operations) {
        //
        // Set the completion block to save the backgroundContext
        //
        if ([operations count] > 0) {
            [[SDCoreDataController sharedInstance] saveBackgroundContext];
        }
        
        //
        // Invoke executeSyncCompletionOperations as this is now the final step of the sync engine's flow
        //
        [self executeSyncCompletedOperations];
    }];
}

Now go to your -processJSONDataRecordsForDeletion method and replace

[self executeSyncCompletedOperations];

with:

[self postLocalObjectsToServer];

Build and run the App! Go ahead and create a new record; create BOTH a Holiday and a Birthday record if you’re feeling brave! :] After the sync finishes, go to the data browser in Parse and you should see your newly created record! It works! This sync stuff is easy; looks like it’s time to fire all the Java guys!