Delays With Automation Rules
Incident Report for Brightpearl
Resolved
We're now confident that our Automation engine has been performing at expected levels for a few weeks and are marking this issue as resolved. Please contact our support team if you have any difficulties.
Posted Jan 09, 2020 - 10:47 UTC
Update
Our recent updates and changes to Automation scheduling have allowed us to significantly clear down the backlog and see a return to typical functionality. As a result we are resetting the scheduling changes we mentioned yesterday. This means orders less than 24 hours old will be checked every 30 minutes again, and new orders up to 6 months old will be monitored at various intervals depending on age - in this latter case please note it will only apply to orders created from now on. Previous orders will stop being monitored after 120 days.

We are aware that new orders are still seeing longer than expected delays (up to 30m) before Automation applies reactive rules, as well as some issues with scheduled rules triggering. We're now focusing on these.
Posted Dec 18, 2019 - 11:27 UTC
Monitoring
We have released a potential solution to this ongoing issue and are monitoring progress against the "backlog" of orders that need to be processed by Automation in both the EU and US regions. We will provide a further update tomorrow following 24 hours of results.
Posted Dec 17, 2019 - 16:32 UTC
Update
We have now identified that significant changes will need to be made to our Automation engine in order to resolve this outstanding issue. As a result we have made the following temporary changes to the frequency with which Automation will "check" an order to see if any rules need to be applied, which we believe will mitigate the current delays some Brightpearl users (in both US and EU regions) are reporting:

The current intervals are:
Order created < 2 days ago - Automation checks every 30 mins
< 31 days - Every 12 hours
< 181 days - Every 24 hours

The new intervals will be:
< 1 day - Every 2 hrs (New orders will be checked immediately upon creation)
< 31 days - Every 12 hrs
< 151 days - Every 24 hrs

We sincerely apologize again for any disruption to your business and thank you for working with us while we attend to this issue. As always, our Support teams are available should you have any questions following this update.
Posted Dec 16, 2019 - 15:12 UTC
Update
Our engineers have identified the issue and are working to resolve it. Please note that all of our US based Automation users will be experiencing delays of varying duration to automated order processing, and we recommend that the batch actions are used to complete orders until further notice. Please contact our support team if you need further guidance on these processes.

We sincerely apologize for the inconvenience at a busy time of year and will do everything we can to assist until the issue has been resolved.
Posted Dec 13, 2019 - 18:02 UTC
Investigating
Our Development Team is still investigating an ongoing issue preventing automation from running on new orders.

At this time, this is only affecting a handful of accounts.
Posted Dec 11, 2019 - 19:20 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Dec 11, 2019 - 16:13 UTC
Investigating
We are experiencing some reports of Automation rules being delayed.

Some users may find that recent orders aren't currently appearing in their Pending list while other rules aren't processing on orders.

We are currently investigating the issue.
Posted Dec 11, 2019 - 13:59 UTC
This incident affected: Brightpearl Applications (Brightpearl Automation).