Author - sapu

New Public Web API

We are excited to announce that we are launching a brand new public web API! If you aren’t sure what this means, you can probably stop reading here and just keep an eye on for new spreadsheets and other out-of-game tools from our favorite content creators and other community members.

Background

We launched the original version of our public API back in early 2016 to allow our users and 3rd party developers to integrate our pricing data in their spreadsheets and other out-of-game tools, websites, and programs. However, we took this version of our public API down at the beginning of Shadowlands as we worked behind the scenes on an extensive (and still ongoing) process of rebuilding all of the backend infrastructure and services which generate TSM pricing data and power our website and desktop application. As part of this, we’ve been working hard on bringing online a new version of our API which leverages this new infrastructure, and we’re finally ready to make it public.

Features

The list of API endpoints we are releasing today covers all the same AuctionDB pricing data you would find in-game from realm-specific min buyout and market value, to region-wide sale prices and rates. This includes both base items and battle pets. This should enable a wide range out-of-game website and tools to leverage TSM’s data to build some great user experiences for gold-makers.

Get Started

Read through the documentation here to get started. We can’t wait to see what you build!

TSM Service Outage March 2022

Actions to Take

Let’s start with the actions you are recommended to take as a TSM user:

  • All users: Update the TSM Desktop Application to the latest version (r410)
    • Your app should update automatically, but you can download it here manually
  • All users: Re-select any Burning Crusade Realms previously configured on the Realm Selection Page for pricing data
  • All users: Enable any Group Notifications previously selected on the Deal Notifications Page

Background

Early in the morning US time on Friday, March 25th, we were notified by our hosting provider that the hard drive containing our primary database died, and they were unable to recover any data from it. After a ton of work, we have since been able to restore all of our services. However, there was some data which we, unfortunately, were not able to recover, which is listed as follows:

  • Premium user addon backups
  • Desktop App realm selections for Burning Crusade realms
  • User configuration data for Group Notifications
  • Some other user account-level and user configuration changes between 4am PDT on March 24th and when our website was brought back online including new accounts and any other changes made on our website during that time

What we Did

We take regular backups of our database and quickly worked to restore from those backups on Friday. Unfortunately, we realized that our backup process had not been updated to include database tables related to Burning Crusade Classic. Additionally, a configuration issue with a script that manages Premium user addon backups also resulted in all backups being lost as a result of this outage.

On Saturday we brought the website back online in a read-only mode as we continued to try to restore some additional database data from a few raw database files which had not been corrupted. After much work, we were successful at restoring some additional data which we otherwise didn’t have proper backups of, but unfortunately, this still left us at the list above of completely lost data.

Naturally, the Premium user addon backups are the most critical. Working with a manifest of the missing files, we modified TSM Desktop Application (version r410) to re-upload any backup files that match the name from the recovered manifest. While we realize this does not guarantee a full recovery of all previously synced backups we have been able to get coverage of a majority of recent backups stored locally to Premium users.

With this in mind, we have set up a dedicated contact address that is available to Premium users who may be missing an important backup that was not captured in the re-upload efforts. We encourage those to reach out to [email protected] and we will explore any available options to make this right.

Additionally, with new resources available along with the protections and processes described in the next section of this post, we have removed the limit on the number of Premium user addon backups that can be marked as ‘saved’ and stored by TSM in the cloud going forward. This means Premium users no longer have to decide which backups to keep available long-term and all backups will be saved indefinitely.

Finally, if you created an account or reset your password on March 24th or March 25th and are having trouble logging in, it is recommended to re-create your account or reset your password again.

Learnings / Improvements

We’ve only had one outage like this in the past, but this one is significantly more serious given the loss of user data. We do take seriously the impact of this event and are working hard to correct things in the short-term, as we discussed above, as well as make immediate changes to prevent this from happening again in the future.

First and foremost, we’ve enabled multiple additional levels of redundancy and protection around our Premium user addon backups. This will prevent them from being irreversibly lost in the future, along with providing a mechanism to quickly recover them if any issues arise. This will have a non-negligible impact on our infrastructure costs, as it is certainly a non-trivial amount of data, especially with the change we’re making to not limit the number of backups we store for people but is the right thing to do given the importance of this data.

Next, we have addressed the gap in our database backup process which resulted in some user configuration data being lost, so any similar event in the future will just be a matter of quickly restoring the backups we have. We’ve also built a number of additional tools to help us recover any lost data in the future, although, of course, we hope we’ll never need to use those.

Lastly, the affected database has been running since 2016. A lot has changed since then in the server infrastructure space, and we’ve been steadily working over the past year to reimplement and migrate our backend services and infrastructure to a more modern, scalable, and maintainable architecture. Most of this has been behind the scenes and is already supporting more user-facing things like Ledger and all of the AuctionDB data which is downloaded by the desktop app, but this outage has definitely raised the importance of moving more things over to this much better system as quickly as possible.

Closing Thoughts

To wrap things up, we’d like to sincerely apologize for the inconvenience caused by this downtime and for the loss of data that occurred. We are looking forward to doing better and further improving the software and services we provide moving forward, and appreciate your continued support and usage of TSM. If you would like to share any thoughts, questions, or feedback – please feel free to share them in the #discussion channel on our Discord server.

Anatomy of a TSM Release: Feature Selection

In this series, we are detailing what goes on behind the scenes to ship a new major TSM addon release. Specifically, we will be describing how version 4.12 of the addon is being built, from deciding what features will be included, to publicly releasing it. This first post will focus on how we decided what features will be in version 4.12 of the addon.

Selection Criteria

There is no end to the list of things we want to implement into TSM to make it the best gold making addon possible. We have curated a list of improvements we are considering on our public roadmap, so any discussion on what we prioritize for the next version starts by reviewing the list there. Our biggest constraint as a team (and, not to get philosophical, but in life as well) is time, but there are many factors we consider as we filter through this list. In no particular order, and with much less formality than they are described here, these include the following high-level points:

  • What portion of our user base would benefit from this feature? How much value will this feature bring to them? How much complexity would it add to the addon for new users or those who wouldn’t directly benefit from it?
  • Is this feature especially relevant to the current state of the game or the other projects being worked on within TSM (i.e. new pricing data functionality)?
  • What resources (design time, development time, dependencies on our web backend or desktop app, etc) are required to ship this feature?

Back when we were deciding what the focus of TSM 4.11 was going to be, we went through a ranking exercise to help us prioritize our list of high-level features we wanted to implement. This resulted in optional material support and custom themes being prioritized for 4.11, and also Gathering being called out as the next big feature to prioritize in 4.12. Therefore, when it came time to plan out 4.12, we went in with the idea of Gathering being the primary focus.

Digging Deeper

Once we’ve decided on the primary feature, in this case Gathering, the next step is to go one level deeper and figure out exactly what use-cases we want to better support. This is almost entirely influenced by feedback we’ve received around Gathering from our users over the years. The most challenging part of this is deciding what we are NOT going to do. Again, we are balancing how much of our time we want to spend on a given feature (and how long before we can release it) vs. how much value it will deliver to our users. For example, Gathering for multiple characters at the same time is not something we are going to be doing in 4.12 because it would require a ton of design and development effort to do it in a way which doesn’t add a ton of complexity that the user needs to manage. It would also provide relatively-low value when compared with the other features, such as supporting intermediate crafts from other professions. The result of this exercise is again reflected in our public roadmap under the “Prioritised Ideas” column. In the case of 4.12, it includes the following:

  • A bunch of general usability and UI improvements to the Task List and Gathering tab of the Crafting UI
  • Adding support for gathering from the bank in Classic
  • Support intermediate crafts on alts as a Gathering source
  • Improved connected realm support
  • Add a default “Farming” source which will track how many the player has acquired through a more manual means

This is a fairly ambitious set of features, but we are fairly confident in being able to tackle them within 4.12. Of course, things may change as we go along, and we may tweak this list as we go through the design and implementation process. However, this list is what we are currently working towards delivering. In addition to the major feature of Gathering, we also went through our overall backlog of feature requests and pulled out a few things which we thought would be easy to implement and/or especially timely to pull into 4.12. This includes things like persisting the “Show Ungrouped Items as” selection between sessions.

Next Steps

The next steps are split between design and development. On the design side, we are brainstorming what a better UI/UX for the task list and Gathering tab looks and feels like. On the development side, we are starting to implement some of the backend changes to support the new features, and knocking out other things which don’t have any dependencies on the new UI.

We are Hiring a JavaScript (React) Developer

UPDATE 3/28: This position has been filled so we are no longer taking applicants for it. Thanks for your interest!

The TradeSkillMaster team is a small group of gamers who love to learn new skills and constantly push the boundaries of what TradeSkillMaster can do for our users. What started as a small addon has grown into a comprehensive gold making addon, desktop application, and set of web-based tools and resources. We are driven by self-improvement and creating a great set of products and services for our users, while working in an efficient, but low-stress environment.

We are always exploring new ways to improve the experience we offer to our users. To that end, we are working on a new version of our desktop application and are looking to add a passionate JS developer to the team to get it across the finish line and shipped to our hundreds of thousands of users.

The Role

As a part of this role, you will be expected to work closely with other members of the team to implement new features, constantly improve the overall quality of the codebase, and drive the project towards initial release and beyond.

This position is part-time (we all have day jobs) and paid hourly. While we are looking for somebody to work on a specific project initially (our new desktop application), this position may be extended beyond that. The work will be purely remote with NA/EU time zones strongly preferred.

About You

Minimum Qualifications:

  • Strong technical background in JS (React preferred) and good code design sense with an emphasis on UI and data processing
  • A proven ability to work in established code bases as part of a team and self-motivation to constantly improve code quality and processes

Preferred Qualifications:

  • The tech stack for this project includes React, Redux. Electron, Socket.IO, Node.js, and Python (backend), so expertise with any or all of those
  • An eagerness to learn about new technologies and jump into different areas of our overall tech stack (from in-game addon to backend infrastructure) as needed
  • Experience with TradeSkillMaster and a passion for improving it

Ready to join us? Send me (Sapu) a message on Discord (https://discord.gg/woweconomy – if I’m AFK I’ll respond ASAP) or send me an email ([email protected]). Come prepared with your resume and any questions about the role you may have.

Ledger Release and Beta Invites

The Ledger section in TSM4 has been in it’s temporary TSM3 placeholder mode until now. We’ve done a lot of work on the section and are proud to share with you the progress we’ve made on the powerful new tool that became of Accounting. The three updated sections you will find in the new Ledger are Revenue, Expenses and Failed Auctions. You will find they feel similar to their TSM3 counterparts, but with the much better look of TSM4.

Ledger

You’ll find that the Inventory viewer isn’t finished yet. We hope to bring it to you soon. Until we do, we hope that you explore the Ledger update, and let us know what you think.

Hopefully you are as excited about the changes we are making to the Ledger in TSM4 as we are. To give these changes in the hands of more goblins as soon as possible we are also sending out a massive new wave of TSM4 beta invites every day this week. If you’ll be one of the lucky goblins joining us in the TSM4 beta, you’ll receive an email from us at the address associated with your TradeSkillMaster account. In this email, you’ll also find a link to an article on our knowledge base that explains how to setup your TSM app to access the beta and how you can join the #beta channel on discord, which is the dedicated place for feedback and questions pertaining to the TSM4 beta.

If you’re not already signed up, go ahead and sign up for the TSM4 beta here. If you like what we do and would like some extra functionality, please consider becoming a TSM premium user. It includes priority access to all of our betas, including TSM4.

Come join our Discord now!

The /r/woweconomy & TSM Discord is growing every day. We’re both incredibly flattered by and proud of the fact that there are now over 23 thousand users. Whether you’re only there to check prices once in awhile, partake in every discussion eagerly, or anything in between, thank you!

If you’re not yet familiar, there are many reasons why you should make the effort. Firstly all TSM support happens in channels dedicated to this purpose. TSM3 question go in the #tradeskillmaster channel, and any issues specific to the TSM4 beta are handled in the #beta channel.

There’s various channels available in the general discussion section too. Here you can discuss your goldmaking methods, or share that neat snipe you just bought! Don’t be afraid to talk with others, try to pick the best channel, and enjoy being a part of this ever growing community!

If you’re only looking to be kept up to date on the latest important news, there is a #news channel for that specific purpose! Large TSM updates, important Blizzard news and particularly important TSM news gets shared in there to offer you a central place.

In short, The /r/woweconomy & TSM Discord is becoming ever larger. It’s got a little bit for everyone, and you too should be a part of it. If you’re still not entirely convinced, simply join and see what you think. There’s even more channels and ways to communicate than just what’s mentioned here!

TSM is Hiring a Part-Time Addon Developer

The TradeSkillMaster team is a small group of gamers who love to learn new skills and constantly push the boundaries of what TradeSkillMaster can do for our users. What started as a small addon has grown into a suite of powerful addons, a desktop application, and an extensive set of web-based tools and resources. We are driven by self-improvement and creating a great set of products and services for our users, while working in an efficient, but laid-back environment.

At TradeSkillMaster, we always strive to improve the products and services we offer. To that end, we are looking for a passionate addon developer to work on adding critical new functionality to our in-game addons which are used by hundreds of thousands of people.

The Role

As a part of this role, you will be expected to work closely with other members of the team to implement exciting new features, constantly improve the overall quality of our codebase, and improve the general stability of the addon.

This position is specifically for somebody who would be working on the TSM addon (TSM4) 25-30 hours a week. Therefore, the ideal candidate would be an independent contractor (potentially with other, smaller workloads), and not somebody with a full-time job. The work would be purely remote, and there are limited requirements on location, but roughly NA/EU timezones are preferred.

About You

Minimum Qualifications:

  • Strong technical background and code design sense with an emphasis on UI and data processing
  • A proven ability to work in established code bases as part of a team and self-motivation to constantly improve code quality and processes

Preferred Qualifications:

  • Lua and WoW addon experience is a plus
  • SQL experience
  • Object-orientated programming
  • Experience with TradeSkillMaster and a passion for improving it

Ready to join us? Send me (Sapu) a message on Discord (https://discord.gg/FJ4877H – if I’m AFK I’ll respond ASAP) or send me an email ([email protected]). Come prepared with your resume and any questions about the role you may have.

Database

This is another very technical blog post which provides a look into the new database library which has been added to the TSM codebase with TSM4. This is an internal code library which makes it easier for us to implement new features in TSM4. This new database library is made possible by the class library which I discussed in a previous blog post.

Structured Data

Our main motivation for creating this database library was to make it easier to represent large datasets in a very structured way which could easily be consumed by the higher-level application logic. Another goal was to be able to connect UI elements to underlying data in a standard way which would allow the UI to update itself automatically when there are changes to the underlying data. Let’s dive into how our database library accomplishes these things. Read More

New TSM4 Beta Version and Invites!

The TSM4 beta is always evolving and today we have some updates to share with you. We’ve been working hard to get TSM4 closer to release. While there is still a lot of work to be done, we’re excited to share our latest progress. There are a few big new features that we’ve been focussing on and will go through in this blog post.

New TSM4 Beta Features

Export

Moving functionality from TSM3 to TSM4 is a big part of the massive project that is TSM4. Importing and exporting of groups and operations are features used by many of our users on a daily basis. There are many enhancements planned for the export feature in TSM4, and we’ve made great progress towards implementing our vision for an awesome export feature. With this latest update, we’ve implemented the brand new TSM4 export UI which makes it as easy as possible to export groups and operations. This is a feature which we will continue to build on and improve as we get closer to the release of TSM4. Read More

Classes

This will be a very technical blog post which provides a look into the new class library which has been added to the TSM codebase with TSM4. This is an internal code library which makes it easier for us to implement new features in TSM4. This new class library is the base for many other internal code libraries in TSM4 such as our UI Builder, Database, and FSM libraries. We will go into more detail on the other libraries in future blog posts.

What is a class?

A common feature of a programming language is being “Object-Oriented”. This means that the structure of the code is based around complex data structures called objects. The type of these objects is generally referred to as their class. The class describes the structure and features of the objects. Read More