Feature Updates

Feature Additions

1. User Created Groups / Lists 

I'm not sure what I want to call it yet, but user curated groups (lists?) currently function as follows. When you are in an athlete's detail view and tap the '+' button at the top right a group selection view will be displayed where you can see all your current groups, and if the athlete you were viewing is in one of those groups a check mark will be next to that group. To add the athlete to a new group or 

2. AutoLayout and iPhone 6 and 6 Plus Support

These aren't major additions for the app because most of my views are table views and text, so they scale with screen dimensions very easily, but I did completely redo the AthleteDetailView with AutoLayout to more easily support larger screen sizes and hopefully make it easier to support landscape orientation in the future (no promises though). 

Bug Fixes

1. Core Data - Read-Only SQLite

I don't fully understand all of the details, but from what I've gathered copying a pre-generated SQLite file from the app bundle to the user documents sandbox doesn't easily handle the multiple files generated while using Write Ahead Logging (WAL), and as a consequence Core Data doesn't like to open SQLite files that were generated with WAL as Read-Only. The most direct solution was to disable the WAL features for BOTH the importer and the app by including the following options string in the NSPersistentStore initialization. 

NSDictionary *options = @{NSSQLitePragmasOption:@{@"journal_mode":@"DELETE"}};

2. Search Crashes

There was a bug that caused the app to crash when you searched>View an athlete>return to the search>started to delete the search string one character at a time. The bug was caused 

Features In Progress

• Server-side Database Loading

Identified Bugs

• User curated groups don't persist after app restarts
• Lines through search results when changing search criteria after viewing an athlete from previous search criteria if the search results didn't fill the display. 

Core Data Import Performance

When creating the PointsStalker data importer I started by following a few different importing tutorials and eventually I got it all figured out. As the datasets I was importing grew from my test batch sizes grew from a few thousand to 18,000+ I realized I was having some scaling problems. The rough numbers said that the import speed was decreasing by a factor of 40x to 60x, so the complete imports were taking from 8 to 10 minutes for 18,000 athletes and their points histories. 

After a few months of tolerating the problem I decided to dig into the issue. Since the initial speed was reasonably good I ruled out the import logic as a major problem, but since the performance degraded over time I thought the problem might have been related to how data was being loaded into memory. In response I did some unscientific parametric testing of the data buffer size and the core data save interval, but the various values had nearly no effect on the overall processing time, so I started to look into other potential problems. 

The next point of interest was an NSMutableDictionary that is used to look up the managed object ID for athletes as new points list data is imported. I thought lookups might be taking a disproportionate amount of time because there are so many athletes to search though, but the points importing was in fact faster than athlete importing despite the points import logic being more complicated.

My gut feeling about NSMutableDictionary lookups was confirmed by an Objc.io article that indicated dictionary lookups should usually be O(1) (constant time). The article also mentioned hashing and creating dictionaries with pre-defined capacity, but concluded that +dictionaryWithCapacity was ineffective. This got me thinking about the performance of adding objects to the NSMutableDictionary and with a little searching I turned up a discussion on Cocoabuilder that explains that as the pre-determined dictionary size is filled to 67% capacity the dictionary is re-hashed and can impact performance. To avoid this the discussion suggests creating the NSMutableDictionary with the underlying CFDictionary class to create a pre-sized dictionary using:

NSMutableDictionary *dict = (NSMutableDictionary*)CFDictionaryCreateMutable(NULL, 1024, &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks);

This change alone cut the import time to 1.5 minutes. To me, that's a huge decrease! Before the CFDictionary change I was wary of making changes that would required regenerating the database, but having MUCH shorter imports has removed that concern. 

(Maybe now I should go back and check out those data buffers and save interval parameters again now that I've eliminated a major limiting factor)