Tracking New vs. Modified Records in EDL Updates

For Enterprise Data License (EDL) customers receiving monthly data overwrites, tracking whether a record is new versus pre-existing and modified can be accomplished using the first_seen_date field in the PERMITS table. While there isn't a specific change flag in the current data schema, this timestamp indicates when Shovels first discovered the permit in our data collection process. By comparing first_seen_date with your previous data delivery, you can identify new records where the first_seen_date falls after your last update. For modifications to existing records, you'll need to implement a comparison between the current and previous data deliveries, focusing on fields like status, valuation, or other attributes that might change over time. This approach requires maintaining a copy of previous deliveries for comparison purposes. We're actively working on enhancing our data delivery system to include explicit change tracking flags in future releases, which will simplify this process for customers who need to monitor specific changes in the permit landscape.