How To Synchronize Core Data with a Web Service – Part 1

Write an AFNetworking client to talk to the Parse REST API

AFNetworking is a class developed by Matt Thompson and Scott Raymond and is described as “a delightful networking library for iOS and Mac OS X.” It makes common tasks like asynchronous http requests a lot easier. As of this writing, AFNetworking does not use ARC, so if you manually add it to a project that is using ARC, the -fno-objc-arc compiler flag is necessary on all of its files. This has already been done for you in the tutorial project.

Note: This tutorial assumes some AFNetworking experience. If you have not used this library before, you should definintely have a read over Getting Started with AFNetworking before you go any further in this tutorial.

The first step to using AFNetworking in your app is to create a client that uses the Singleton pattern. Go to File\New\File…, choose iOS\Cocoa Touch\Objective-C class, and click Next. Enter AFHTTPClient for Subclass of, name the new class SDAFParseAPIClient, click Next and Create.

Open your interface file, SFAFParseAPIClient.h and add a new class method:

#import "AFHTTPClient.h"

@interface SDAFParseAPIClient : AFHTTPClient

+ (SDAFParseAPIClient *)sharedClient;

@end

And then complete the implementation in SFAFParseAPIClient.m:

#import "SDAFParseAPIClient.h"

static NSString * const kSDFParseAPIBaseURLString = @"https://api.parse.com/1/";

static NSString * const kSDFParseAPIApplicationId = @"YOUR_APPLICATION_ID";
static NSString * const kSDFParseAPIKey = @"YOUR_REST_API_KEY";

@implementation SDAFParseAPIClient

+ (SDAFParseAPIClient *)sharedClient {
    static SDAFParseAPIClient *sharedClient = nil;
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{
        sharedClient = [[SDAFParseAPIClient alloc] initWithBaseURL:[NSURL URLWithString:kSDFParseAPIBaseURLString]];
    }); 

    return sharedClient;
}

@end

Note that in the code above, you need to insert the personal information generated by Parse: YOUR_APPLICATION_ID and YOUR_REST_API_KEY are unique to your app. Replace YOUR_APPLICATION_ID and YOUR_REST_API_KEY with the values from the Overview tab of the Parse project window.

It also creates three static NSString variables for the Parse API URL, your Parse API Application Id, and your Parse API Key, and implements the +sharedClient method which uses GCD to create a new instance of the class and store its reference in a static variable, thus becoming a Singleton.

Next import AFJSONRequestOperation.h in SDAFParseAPIClient.m:

#import "AFJSONRequestOperation.h"

Then override -initWithBaseURL: in SDAFParseAPIClient.m to set the parameter encoding to JSON and initialize the default headers to include your Parse Application ID and Parse API Key:

- (id)initWithBaseURL:(NSURL *)url {
    self = [super initWithBaseURL:url];
    if (self) {
        [self registerHTTPOperationClass:[AFJSONRequestOperation class]];
        [self setParameterEncoding:AFJSONParameterEncoding];
        [self setDefaultHeader:@"X-Parse-Application-Id" value:kSDFParseAPIApplicationId];
        [self setDefaultHeader:@"X-Parse-REST-API-Key" value:kSDFParseAPIKey];
    }

    return self;

}

Add two methods to your interface for SDAFParseAPIClient in SDAFParseAPIClient.h:

- (NSMutableURLRequest *)GETRequestForClass:(NSString *)className parameters:(NSDictionary *)parameters;
- (NSMutableURLRequest *)GETRequestForAllRecordsOfClass:(NSString *)className updatedAfterDate:(NSDate *)updatedDate;

Beneath -initWithBaseURL in SDAFParseAPIClient.m, implement the two methods -GETRequestForClass:parameters: and -GETRequestForAllRecordsOfClass:updatedAfterDate:.

- (NSMutableURLRequest *)GETRequestForClass:(NSString *)className parameters:(NSDictionary *)parameters {
    NSMutableURLRequest *request = nil;
    request = [self requestWithMethod:@"GET" path:[NSString stringWithFormat:@"classes/%@", className] parameters:parameters];
    return request;
}

- (NSMutableURLRequest *)GETRequestForAllRecordsOfClass:(NSString *)className updatedAfterDate:(NSDate *)updatedDate {
    NSMutableURLRequest *request = nil;
    NSDictionary *parameters = nil;
    if (updatedDate) {
        NSDateFormatter *dateFormatter = [[NSDateFormatter alloc] init];
        [dateFormatter setDateFormat:@"yyyy-MM-dd'T'HH:mm:ss.'999Z'"];
        [dateFormatter setTimeZone:[NSTimeZone timeZoneWithName:@"GMT"]];

        NSString *jsonString = [NSString 
                                stringWithFormat:@"{\"updatedAt\":{\"$gte\":{\"__type\":\"Date\",\"iso\":\"%@\"}}}", 
                                [dateFormatter stringFromDate:updatedDate]];

        parameters = [NSDictionary dictionaryWithObject:jsonString forKey:@"where"];
    }

    request = [self GETRequestForClass:className parameters:parameters];
    return request;
}

-GETRequestForClass:parameters: will return an NSMutableURLRequest used to GET records from the Parse API for a Parse Object with the class name ‘className’ and submit an NSDictionary of parameters. Acceptable parameters can be seen in the Parse REST API Documentation.

-GETRequestForAllRecordsOfClass:updatedAfterDate: will return an NSMutableURLRequest used to GET records from the Parse API that were updated after a specified NSDate. Notice that this method creates the parameters dictionary and calls your other method. This is merely a convenience method so that the parameters dictionary does not have to be generated each time a request is made using a date.

Okay! So you now have an AFNetworking client ready to go. But it’s not much good until you get the data synchronized! The section below will get you there.

Create a “Sync Engine” Singleton class to handle synchronization

Add another new Singleton class to manage all of the synchronization routines between Core Data and your remote service (Parse). Go to File\New\File…, choose iOS\Cocoa Touch\Objective-C class, and click Next. Enter NSObject for Subclass of, name the new class SDSyncEngine, click Next and Create.

#import <Foundation/Foundation.h>

@interface SDSyncEngine : NSObject

+ (SDSyncEngine *)sharedEngine;

@end

Add a static method +sharedEngine to access the Singleton’s instance.

#import "SDSyncEngine.h"

@implementation SDSyncEngine

+ (SDSyncEngine *)sharedEngine {
    static SDSyncEngine *sharedEngine = nil;
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{
        sharedEngine = [[SDSyncEngine alloc] init];
    });

    return sharedEngine;
}

@end

In order to synchronize data between Core Data (your local records) and Parse (the server-side records), you will use a strategy where NSManagedObject sub-classes are registered with SDSyncEngine. The sync engine will then handle the necessary process to take data from Parse, and…uh…parse it (for lack of a better term!), and save it to Core Data.

Declare a new method in SDSyncEngine.h to register classes with the sync engine:

- (void)registerNSManagedObjectClassToSync:(Class)aClass;

Add a “private” category in SDSyncEngine.m with a property to store all of the registered classes and synthesize:

@interface SDSyncEngine ()

@property (nonatomic, strong) NSMutableArray *registeredClassesToSync;

@end

@implementation SDSyncEngine

@synthesize registeredClassesToSync = _registeredClassesToSync;

...

Beneath +sharedEngine, add the implementation:

- (void)registerNSManagedObjectClassToSync:(Class)aClass {
    if (!self.registeredClassesToSync) {
        self.registeredClassesToSync = [NSMutableArray array];
    }

    if ([aClass isSubclassOfClass:[NSManagedObject class]]) {        
        if (![self.registeredClassesToSync containsObject:NSStringFromClass(aClass)]) {
            [self.registeredClassesToSync addObject:NSStringFromClass(aClass)];
        } else {
            NSLog(@"Unable to register %@ as it is already registered", NSStringFromClass(aClass));
        }
    } else {
        NSLog(@"Unable to register %@ as it is not a subclass of NSManagedObject", NSStringFromClass(aClass));
    }

}

This method takes in a Class, initializes the registeredClassesToSync property (if it is not already), verifies that the object is a subclass of NSManagedObject, and, if so, adds it to the registeredClassesToSync array.

Note: It’s always preferable to write efficient code, but when it comes to synchronizing data with an online service, you want to get the most “bang for the buck” with each synchronization call. The Parse service may have a free tier, but that doesn’t mean it’s unlimited – and you want to make use of every call in the most efficient way possible! Plus keep in mind that every piece of data pulled over the mobile network counts against the user’s data plan. No one will want to use your app if it’s going to rack up excess data charges! :]

One main concern with keeping the data synchronized is doing it efficiently, so it doesn’t make sense to download and process every record each time the sync process is executed. One solution is to use a process generally known as performing a “delta sync”, meaning “only give me the new stuff, I don’t care about what I already know”.

Your delta sync process will be accomplished by looking at the “updatedAt” attribute on your Entities and determining which one is the most recent. This date will then be used to ask the remote service to only return records who were modified after this date.

Import SDCoreDataController.h in SDSyncEngine.m:

#import "SDCoreDataController.h"

Then add this new method:

- (NSDate *)mostRecentUpdatedAtDateForEntityWithName:(NSString *)entityName {
    __block NSDate *date = nil;
    // 
    // Create a new fetch request for the specified entity
    //
    NSFetchRequest *request = [NSFetchRequest fetchRequestWithEntityName:entityName];
    //
    // Set the sort descriptors on the request to sort by updatedAt in descending order
    //
    [request setSortDescriptors:[NSArray arrayWithObject:
                                 [NSSortDescriptor sortDescriptorWithKey:@"updatedAt" ascending:NO]]];
    //
    // You are only interested in 1 result so limit the request to 1
    //
    [request setFetchLimit:1];
    [[[SDCoreDataController sharedInstance] backgroundManagedObjectContext] performBlockAndWait:^{
        NSError *error = nil;
        NSArray *results = [[[SDCoreDataController sharedInstance] backgroundManagedObjectContext] executeFetchRequest:request error:&error];
        if ([results lastObject])   {
            //
            // Set date to the fetched result
            //
            date = [[results lastObject] valueForKey:@"updatedAt"];
        }
    }];

    return date;
}

This returns the “most recent last modified date” for a specific entity.

Next add another new method downloadDataForRegisteredObjects: beneath mostRecentUpdatedAtDateForEntityWithName:

#import "SDAFParseAPIClient.h"
#import "AFHTTPRequestOperation.h"

In SDSyncEngine.h import SDAFParseAPIClient.h and AFHTTPRequestOperation.h.

- (void)downloadDataForRegisteredObjects:(BOOL)useUpdatedAtDate {
    NSMutableArray *operations = [NSMutableArray array];

    for (NSString *className in self.registeredClassesToSync) {
        NSDate *mostRecentUpdatedDate = nil;
        if (useUpdatedAtDate) {
            mostRecentUpdatedDate = [self mostRecentUpdatedAtDateForEntityWithName:className];
        }
        NSMutableURLRequest *request = [[SDAFParseAPIClient sharedClient] 
                                        GETRequestForAllRecordsOfClass:className 
                                        updatedAfterDate:mostRecentUpdatedDate];
        AFHTTPRequestOperation *operation = [[SDAFParseAPIClient sharedClient] HTTPRequestOperationWithRequest:request success:^(AFHTTPRequestOperation *operation, id responseObject) {
            if ([responseObject isKindOfClass:[NSDictionary class]]) {
                NSLog(@"Response for %@: %@", className, responseObject);
                // 1
                // Need to write JSON files to disk

            }            
        } failure:^(AFHTTPRequestOperation *operation, NSError *error) {
            NSLog(@"Request for class %@ failed with error: %@", className, error);
        }];

        [operations addObject:operation];
    }

    [[SDAFParseAPIClient sharedClient] enqueueBatchOfHTTPRequestOperations:operations progressBlock:^(NSUInteger numberOfCompletedOperations, NSUInteger totalNumberOfOperations) {

    } completionBlock:^(NSArray *operations) {
        NSLog(@"All operations completed");
        // 2
        // Need to process JSON records into Core Data
    }];
}

This method iterates over every registered class, creates NSMutableURLRequests for each, uses those requests to create AFHTTPRequestOperations, and finally at long last passes those operations off to the -enqueueBatchOfHTTPRequestOperations:progressBlock:completionBlock method of SDAFParseAPIClient.

Notice that this method is not complete! Take a look at Comment 1 inside the success block for the AFHTTPRequestOperation. You will later add a method in this block that takes the response received from the remote service and saves it to disk. Now check out Comment 2; this block will be called when all operations have completed. You’ll later add a method here that takes the responses saved to disk and processes them into Core Data.

Awesome! You’ve written a ton of code already! You’re almost to the point where you’ll start seeing some progress in the application. However, at this point you’re lacking a way to start the whole sync process — which is the whole point of this app! :] But tread carefully – the manner in which the sync process is started is crucial, and it’s important to keep track of the status, as you do not want to start the sync process more than once.

Add a readonly property to SDSyncEngine.h to track the sync status:

@property (atomic, readonly) BOOL syncInProgress;

Synthesize the syncInProgress property:

@synthesize syncInProgress = _syncInProgress;

Declare a -startSync method in SDSyncEngine.h:

- (void)startSync;

And add it’s implementation in SDSyncEngine.m:

- (void)startSync {
    if (!self.syncInProgress) {
        [self willChangeValueForKey:@"syncInProgress"];
        _syncInProgress = YES;
        [self didChangeValueForKey:@"syncInProgress"];
        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
            [self downloadDataForRegisteredObjects:YES];
        });
    }
}

You’ve implemented the -startSync method which first checks if the sync is already in progress, and if not, sets the syncInProgress property. It then uses GCD to kick off an asynchronous block that calls your downloadDataForRegisteredObjects: method.

Moving right along, you need to register your NSManagedObject classes and start the sync!Import the appropriate classes in SDAppDelegate.m:

#import "SDSyncEngine.h"
#import "Holiday.h"
#import "Birthday.h"

And then add this to application:didFinishLaunchingWithOptions:

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
    [[SDSyncEngine sharedEngine] registerNSManagedObjectClassToSync:[Holiday class]];
    [[SDSyncEngine sharedEngine] registerNSManagedObjectClassToSync:[Birthday class]];

    return YES;
}

This registers the Holiday and Birthday classes with the sync engine in -application:didFinishLaunchingWithOptions:; this pattern allows you to easily add other objects to the sync engine in the future, should you wish to extend the application! Scalability is always good! :]

Then call startSync in applicationDidBecomeActive:

- (void)applicationDidBecomeActive:(UIApplication *)application
{
    [[SDSyncEngine sharedEngine] startSync];
}

You made it! You’ve been quite a trooper getting to this point — and yes, you can build and run the App! In your Xcode or device console you’ll see something very close to the following:

2012-07-09 00:39:15.764 SignificantDates[70812:fb03] Response for Holiday: {
    results =     (
                {
            createdAt = "2012-07-09T07:13:24.593Z";
            date =             {
                "__type" = Date;
                iso = "2012-12-25T00:00:00.000Z";
            };
            details = "Give gifts";
            image =             {
                "__type" = File;
                name = "9d2d8a0d-36fb-4abe-9908-bebd7fb39056-christmas.gif";
                url = "http://files.parse.com/bcee5dd3-46dc-40a8-abe6-37da2732e809/9d2d8a0d-36fb-4abe-9908-bebd7fb39056-christmas.gif";
            };
            name = Christmas;
            objectId = FVkYM9QROH;
            observedBy =             (
                US,
                UK
            );
            updatedAt = "2012-07-09T07:36:28.097Z";
        }
    );
}
2012-07-09 00:39:15.765 SignificantDates[70812:fb03] Response for Birthday: {
    results =     (
                {
            createdAt = "2012-07-09T07:34:39.745Z";
            date =             {
                "__type" = Date;
                iso = "2012-11-01T00:00:00.000Z";
            };
            facebook = NicoleSn00kiPolizzi;
            giftIdeas = "A brain";
            image =             {
                "__type" = File;
                name = "5dcc3de5-3add-466a-bb46-31a7b7115903-nicole-polizzi.jpg";
                url = "http://files.parse.com/bcee5dd3-46dc-40a8-abe6-37da2732e809/5dcc3de5-3add-466a-bb46-31a7b7115903-nicole-polizzi.jpg";
            };
            name = "Nichole (Snooki) Polizzi";
            objectId = 23S04NSPOR;
            updatedAt = "2012-07-09T07:36:11.792Z";
        }
    );
}
2012-07-09 00:39:15.767 SignificantDates[70812:fb03] All operations completed

Exciting stuff! Now it’s time to do something with this data; it’s just floating around, and you need to persist this data to local storage. The key concept in data transactions is to perform as many network operations as possible in a single batch, in order to reduce network traffic. A really easy way to accomplish this is to queue all requests and save the responses off to disk before processing them.

Add these three three methods to SDSyncEngine.m (beneath downloadDataForRegisteredObjects:) to handle file management:

#pragma mark - File Management

- (NSURL *)applicationCacheDirectory
{
    return [[[NSFileManager defaultManager] URLsForDirectory:NSCachesDirectory inDomains:NSUserDomainMask] lastObject];
}

- (NSURL *)JSONDataRecordsDirectory{

    NSFileManager *fileManager = [NSFileManager defaultManager];
    NSURL *url = [NSURL URLWithString:@"JSONRecords/" relativeToURL:[self applicationCacheDirectory]];
    NSError *error = nil;
    if (![fileManager fileExistsAtPath:[url path]]) {
        [fileManager createDirectoryAtPath:[url path] withIntermediateDirectories:YES attributes:nil error:&error];
    }

    return url;
}

- (void)writeJSONResponse:(id)response toDiskForClassWithName:(NSString *)className {
    NSURL *fileURL = [NSURL URLWithString:className relativeToURL:[self JSONDataRecordsDirectory]];
    if (![(NSDictionary *)response writeToFile:[fileURL path] atomically:YES]) {
        NSLog(@"Error saving response to disk, will attempt to remove NSNull values and try again.");
        // remove NSNulls and try again...
        NSArray *records = [response objectForKey:@"results"];
        NSMutableArray *nullFreeRecords = [NSMutableArray array];
        for (NSDictionary *record in records) {
            NSMutableDictionary *nullFreeRecord = [NSMutableDictionary dictionaryWithDictionary:record];
            [record enumerateKeysAndObjectsUsingBlock:^(id key, id obj, BOOL *stop) {
                if ([obj isKindOfClass:[NSNull class]]) {
                    [nullFreeRecord setValue:nil forKey:key];
                }
            }];
            [nullFreeRecords addObject:nullFreeRecord];
        }

        NSDictionary *nullFreeDictionary = [NSDictionary dictionaryWithObject:nullFreeRecords forKey:@"results"];

        if (![nullFreeDictionary writeToFile:[fileURL path] atomically:YES]) {
            NSLog(@"Failed all attempts to save response to disk: %@", response);
        }
    }
}

The first two methods return an NSURL to a location on disk where the files will reside. The third is more specific to the application and the remote service; each response is saved to disk as its respective class name.

One interesting situation with the Parse API is that it will return values, which translate to NSNull objects when converted to an NSDictionary. Unfortunately, it is not possible to serialize an NSNull object to disk — you can’t save what is, essentially, nothing! Therefore, you must remove all of the NSNull objects before persisting your data.

You’ll take an optimistic approach here when persisting your data. You’ll attempt to same the response first, without scanning for NSNull objects. If that attempt fails, then scan for NSNull objects, remove any that are found, and try again. If THAT attempt fails, then all you can do is to fall back to standard error handling techniques, where you alert the user or report the issue. this tutorial won’t cover those error handling operations, but you can easily add your own if you so desire.

Next it’s time to modify downloadDataForRegisteredObjects – replace the placeholder comment 1 as follows:

...

- (void)downloadDataForRegisteredObjects:(BOOL)useUpdatedAtDate {
    NSMutableArray *operations = [NSMutableArray array];
    
    for (NSString *className in self.registeredClassesToSync) {
        NSDate *mostRecentUpdatedDate = nil;
        if (useUpdatedAtDate) {
            mostRecentUpdatedDate = [self mostRecentUpdatedAtDateForEntityWithName:className];
        }
        NSMutableURLRequest *request = [[SDAFParseAPIClient sharedClient]
                                        GETRequestForAllRecordsOfClass:className
                                        updatedAfterDate:mostRecentUpdatedDate];
        AFHTTPRequestOperation *operation = [[SDAFParseAPIClient sharedClient] HTTPRequestOperationWithRequest:request success:^(AFHTTPRequestOperation *operation, id responseObject) {
            if ([responseObject isKindOfClass:[NSDictionary class]]) {
                [self writeJSONResponse:responseObject toDiskForClassWithName:className];
            }
        } failure:^(AFHTTPRequestOperation *operation, NSError *error) {
            NSLog(@"Request for class %@ failed with error: %@", className, error);
        }];
        
        [operations addObject:operation];
    }
    
    [[SDAFParseAPIClient sharedClient] enqueueBatchOfHTTPRequestOperations:operations progressBlock:^(NSUInteger numberOfCompletedOperations, NSUInteger totalNumberOfOperations) {
        
    } completionBlock:^(NSArray *operations) {
        NSLog(@"All operations completed");
        // 2
        // Need to process JSON records into Core Data
    }];
}

You’ve replaced the comment with a much more useful action — writing the data to file! The NSLog goes away, and the -writeJSONResponse:toDiskForClassWithName: method is called.

Build and run your App! You’ll see that the files are saved to disk.

If you want to see the results of all your hard work, you can view the actual files by opening Finder, open the Go menu, choose “Go to Folder…” and enter ‘~/Library/Application Support/iPhone Simulator/’. (If you prefer to click through all the folders, hold down “option” when selecting Go in Finder and Library will reappear in the list of possible destinations.) From here, you will need to do some detective work!

First, determine which version of the simulator you are running. Open the appropriate folder for that simulator, then open Applications. Next you will likely see a number of folders with random names. Eek! Stay cool. An easy way to find the correct folder is to sort the folders by Date Modified and open the most recently modified folder. You will know you found the correct folder once you see “SignificantDates.app” in the folder.

Once you are in the correct App folder, open Library > Caches > JSONRecords. Here you will see the Holiday and Birthday files. You can open the files with Xcode to view them as they are simple Property List files. Hooray! If you see your data in the files, then you know your app is working! :]

At this point the sync process is finished! Even though it doesn’t do a whole lot right now, you still need to be aware when syncing is in progress, and when it is not. You already have a BOOL to track this, but you need to set it to NO at this point in order to stop sync. You’ll also want to know when the App is syncing for the first time, otherwise known as the initial sync. To track this information add the following @interface SDSyncEngine() in SDSyncEngine.m:

NSString * const kSDSyncEngineInitialCompleteKey = @"SDSyncEngineInitialSyncCompleted";
NSString * const kSDSyncEngineSyncCompletedNotificationName = @"SDSyncEngineSyncCompleted";

Then add these methods above -mostRecentUpdatedAtDateForEntityWithName:

- (BOOL)initialSyncComplete {
    return [[[NSUserDefaults standardUserDefaults] valueForKey:kSDSyncEngineInitialCompleteKey] boolValue];
}

- (void)setInitialSyncCompleted {
    [[NSUserDefaults standardUserDefaults] setValue:[NSNumber numberWithBool:YES] forKey:kSDSyncEngineInitialCompleteKey];
    [[NSUserDefaults standardUserDefaults] synchronize];
}

- (void)executeSyncCompletedOperations {
    dispatch_async(dispatch_get_main_queue(), ^{
        [self setInitialSyncCompleted];
        [[NSNotificationCenter defaultCenter] 
         postNotificationName:kSDSyncEngineSyncCompletedNotificationName 
         object:nil];
        [self willChangeValueForKey:@"syncInProgress"];
        _syncInProgress = NO;
        [self didChangeValueForKey:@"syncInProgress"];
    });
}

After -startSync add the method above which will be called when the sync process finishes.