Records missing from algolia index after runnning laravel scout import

Harish Toshniwal • February 24, 2020

original laravel

You run php artisan scout:import and it says [OK] All x-y-z records have been imported. You then go to your algolia index and see that some of the records are missing. This can be a really confusing issue to fix.

If the value of the queue option in the config/scout.php configuration file is set to true, we aren't notified of any errors that arise during the batch import i.e. while runnning the php artisan scout:import command. The command output at the end gives a sense of false positive saying [OK] All x-y-z records have been imported. The errors won't even have to be in our codebase, it can be algolia rejecting the model/object saying "the object size is too big, request for a larger quota" if a single random record is few Kb's bigger. Once such error happens, no records after that are imported to algolia.

Lemme explain with a quick example: If there are 1000 records and you run php artisan scout:import, if the record number 567 is bigger than the object size allowed by your algolia plan, then the records 568 to 1000 won't be synced to algolia. All this because the import was queued and there was no way to notify the user about the error. To debug the issue you should temporarily set the queue option to false and re-run the import command.

I have created a PR on the laravel scout docs to add a tip about this possible issue. Hopefully, this blog saved you some time fixing this weird issue we spent couple of days to figure out.

If you are aware of other possible & more better solutions, you can correct me with those on twitter.