Expanding Unsplash Image Coverage: A Comprehensive Guide

This article delves into expanding Unsplash image coverage within an application, focusing on strategies to improve image availability for various locations, optimize refresh processes, and enhance overall user experience. Currently, Unsplash images are downloaded only for cities marked with discovery_enabled = true, which presents several limitations. This guide proposes solutions to address these issues and ensure a more comprehensive and efficient image delivery system.

Problem Summary

Currently, the system's image coverage is constrained by a few critical issues:

1. Limited City Coverage: Existing scrapers, such as Geeks Who Drink, gather venue data across numerous cities. However, the application downloads Unsplash images only for cities with discovery_enabled = true. This selective approach leads to a fragmented user experience, as many cities with listed venues lack corresponding images. This limitation means that users browsing these cities encounter a less visually rich and engaging experience, potentially diminishing their overall satisfaction with the application. It's crucial to address this gap to ensure consistency and high-quality user experience across all locations.

2. Lack of Country Image Support: The absence of an unsplash_gallery field in the countries table prevents the display of country-specific images within the app. This omission represents a significant gap in the application's visual content, particularly for users interested in exploring countries or regions. Without country images, the application misses an opportunity to provide a more immersive and informative experience, potentially limiting user engagement and exploration. Adding country image support is essential to create a more holistic and visually appealing platform.

3. Inefficient Refresh Strategy: The system currently refreshes images daily at 3 AM UTC for all active cities, regardless of the last_refreshed_at timestamp. This approach is inefficient, as Unsplash images do not require such frequent updates. The unnecessary daily refreshes consume system resources and API quota without providing significant value. A smarter refresh strategy, which considers the staleness of existing images, would optimize resource utilization and reduce the load on the Unsplash API.

4. API Rate Limit Wastage: The current implementation makes five API calls per city daily, leading to a wasteful consumption of the Unsplash rate limit quota. Unsplash allows 5000 requests per hour, but the daily refresh strategy inefficiently utilizes this quota. By implementing a more intelligent approach, such as staleness checking, the application can drastically reduce the number of API calls, preserving the quota for other potential uses and ensuring the system remains within its operational limits. Optimizing API usage is critical for maintaining the application's long-term performance and scalability.

Current Implementation

The current implementation has distinct features for managing city images but lacks support for countries. Here’s a breakdown:

Cities

• Schema: The cities.unsplash_gallery (JSONB) field exists, which is a positive foundation. ✅
• Categories: There are five categories for city images: general, architecture, historic, old_town, and city_landmarks.
• Images: The system fetches ten images per category, resulting in 50 images per city.
• Tracking: A last_refreshed_at timestamp is stored per category, indicating the last time images for that category were updated. ✅
• Criteria: Images are downloaded only for cities where discovery_enabled = true. ❌
• Refresh: Images are refreshed daily at 3 AM UTC via UnsplashRefreshWorker. ❌
• Worker: The UnsplashCityRefreshWorker handles the refresh process within the :unsplash Oban queue.

Countries

• Schema: No unsplash_gallery field exists in the countries table. ❌
• Images: There is no support for country images in the current system. ❌

Proposed Solutions

To address the identified problems, several solutions are proposed, focusing on expanding city image criteria, implementing smart staleness checking, adding country image support, and configuring refresh intervals.

1. Expand City Image Criteria

Current: Images are only downloaded for cities with discovery_enabled = true.

Proposed: Expand the criteria to include cities with venue_count >= 3.

Rationale:

• Scrapers often add venues to cities beyond those designated as