For the past two weeks I have been using various models offered by amazon bedrock to develop the task management system for wayfinder.
During the first week, I realized that running the model locally would not be feasible. The application was incredibly resource intensive and the response time for breaking down a single task took upwards of five minutes, and thats if the application didn’t crash during the process.
In light of this, I decided to ditch the “offline” aspect of this project as in addition to not being technically feasible, I feel that it does not make logistical sense considering the goals of this project.
Wayfinder is a dynamic task management application designed for the workplace. A user would very likely need to be connected to the internet to submit their tasks/assignments anyway, so having the operate offline does not benefit the user in any material sense. In its current state, having the application operate entirely offline would act as a hinderance for the user.
During the second week, I moved towards accessing FMs through AWS. Using FMs through AWS offered alot of benefits, like an accessible UI for response management.
One downside with using AWS is rate limiting. Each of the models I used had response throttling depending on if the user sent to many messages or if the responses themselves were too long.
To get around this I had to set several guardrails in AWS to ensure I was able to get responses consistently.
For the visualization portion of this, I ran into sever problems where task blocks would overlap one another or repeat infinitely off of the calendar itself. I was unable to find the root cause for this, so In the interest of time I decided to just use a simple visualization script to handle the calendar.
Leave a comment