Project URL: https://github.com/steemit/steemdb

Here is the report generated by AI.

---

Changelog

Release Notes - November 2025

🚨 Important Infrastructure Updates

MongoDB Infrastructure Upgrade

We have upgraded our MongoDB infrastructure to improve performance and reliability:

• EC2 Instance Upgrade: Migrated from t3.xlarge to m5.2xlarge

  • Better CPU performance and network throughput
  • Improved memory allocation for database operations

• MongoDB Version Upgrade: Upgraded from MongoDB 4.4 to MongoDB 8.x

  • Latest features and security improvements
  • Enhanced query performance and indexing capabilities
  • Better support for modern aggregation pipelines

⚠️ Database Status: During the upgrade process, the database encountered corruption issues. We are currently re-syncing the entire database from the blockchain. This process is expected to take 1-2 weeks to complete. Some historical data may be temporarily unavailable during this period.

---

✨ New Features

Account History Infinite Scroll (Waterfall)

• Added: Infinite scroll functionality for account history pages
  • Automatically loads more history records when scrolling down 75% of the page
  • Throttled loading (500ms throttle) to prevent excessive API calls
  • Smooth pagination with loading indicators
  • Improved user experience for browsing account activity

Enhanced API Endpoints

• New API: /api/history/{account} - Account history waterfall API
  • Supports pagination with start and limit parameters
  • Returns structured JSON with data, nextStart, and hasMore fields
  • Improved error handling and validation

---

🐛 Bug Fixes

Account Pages

• Fixed: Account name not available in chart templates when account doesn't exist in database

  • Changed all chart templates to use accountName instead of account.name
  • Ensures charts work correctly even when account data is not yet in database

• Fixed: Route conflict between /api/account/{account} and /api/account/{account}/{action}

  • Reordered routes to prioritize more specific patterns first
  • Fixed route parameter extraction for account names

• Fixed: Fatal error when account not found in viewAction()

  • Added proper null checking before calling toArray()
  • Returns appropriate error messages instead of crashing

Chart Data Validation

• Fixed: Plottable.js errors when chart data is empty or invalid
  • Added data validation and filtering before creating Plottable datasets
  • Safe accessor functions that handle null/undefined values
  • Prevents chart rendering errors when API returns empty data

Comments and Forums

• Fixed: Comment not exist error handling
• Fixed: Accounts page display issues
• Fixed: Post count zero issue

---

⚡ Performance Improvements

Data Synchronization Optimization

• Concurrent Processing: Implemented thread pool for parallel RPC calls

  • Comment processing: up to 10 concurrent workers
  • Account processing: up to 5 concurrent workers
  • Result: 5-10x faster processing (depending on network latency)

• Database Query Optimization:

  • Removed unnecessary count_documents() queries
  • Replaced with len() operations on already-fetched lists
  • Reduced database query overhead

• Data Processing Optimization:

  • Optimized update_comment() function data processing
  • Pre-defined date format strings to avoid repeated creation
  • Batch processing of type conversions
  • Result: Reduced CPU usage during data transformation

Steemd API Integration

• Improved: Changed from get_account_history to condenser_api.get_account_history
  • Better compatibility with Steem API
  • Added proper integer type casting for parameters
  • Enhanced error logging for debugging

---

🔧 Technical Improvements

Code Quality

• Documentation: All Chinese comments translated to English
  • Improved code readability for international developers
  • Consistent English documentation throughout codebase

Error Handling

• Enhanced: Better exception handling across controllers
  • Individual task exceptions caught and logged without stopping entire processes
  • Progress tracking with completion counts for monitoring
  • More resilient error recovery

Docker & Infrastructure

• Updated: Docker Python base image downgraded for Steem/pycrypto compatibility
• Added: Architecture analysis and MongoDB CPU optimization documentation
• Improved: Sync script with better error handling and progress reporting

---

📝 Documentation

• Added: ARCHITECTURE_ANALYSIS.md - Architecture selection analysis
• Added: MONGODB_CPU_OPTIMIZATION.md - MongoDB CPU optimization guidelines
• Added: OPTIMITION_PARAMS.md - Optimization parameters documentation
• Added: docker/sync/CHANGELOG.md - Sync service changelog

---

🔄 API Changes

Modified Endpoints

• /api/account/{account} - Now validates account name and handles missing accounts gracefully
• /api/account/{account}/{action} - Route priority fixed for proper action routing
• /api/history/{account} - New endpoint for account history pagination

Response Format Changes

• All API errors now return proper JSON error objects
• Added debug field in some responses for troubleshooting (temporary)
• Improved error messages and validation

---

📊 Statistics

• Files Changed: 33 files
• Lines Added: ~1,510 insertions
• Lines Removed: ~111 deletions
• Net Change: +1,399 lines

---

⚠️ Known Issues & Limitations

1. Database Re-sync in Progress: Historical data may be incomplete during the 1-2 week re-sync period
2. Chart Loading: Some charts may show empty data if account history is not yet synchronized
3. API Rate Limiting: Concurrent worker limits are set to prevent overwhelming RPC servers
   • May need adjustment based on server capacity

---

🚀 Migration Notes

• No Breaking Changes: All changes are backward compatible
• No Configuration Required: No manual configuration changes needed
• Database Schema: No database schema changes required
• Dependencies: All existing dependencies remain compatible

---

🙏 Acknowledgments

Thank you for your patience during the infrastructure upgrade and database re-synchronization process. We appreciate your continued support as we work to restore full functionality.

---

For support or to report issues, please visit our GitHub repository or contact the development team.