MastodonContentMover

• What is MastodonContentMover?
• When should I use it?
• How do I use it?
• Parameter guide
• Example commands
• Security
• Information for instance administrators
• Further information
• Acknowledgements

---

Important

MastodonContentMover is currently in development. The current, early-stage development testing version is available for development testing purposes only. It is likely to fail in ungainly and/or unexpected ways, produce ugly error messages, or maybe just plain not work at all. Please be aware that you use it at your own risk. The authors of this tool accept no responsibility for any data undesirably lost or created through its use, or any ill effects due to its misuse.

---

What's new?

Development release v0.01.06: Adds parameter to allow disabling of idempotency key for testing purposes, applies latest Bigbone library snapshot and increases the value of the extra throttle used when overriding default settings to repost statuses to public timelines. (version history)

Last update Saturday September 2, 2023

---

What is MastodonContentMover?

MastodonContentMover is a command-line tool that downloads your posts from one Mastodon instance, saving them as a set of files on your computer, and then re-posts them on any other Mastodon instance.

Its purpose is to allow Mastodon users to move content they value when migrating from one instance to another, which is not currently possible within Mastodon itself.

What can it do?

It can:

• save the text of your posts and any attached media from a source Mastodon instance to your computer
• re-post new copies of those posts, including the text and any attached media, from your computer to another Mastodon instance
• save "alt text" descriptions of images and video when saving media, and reapply them when media is re-posted
• save image focal points, and reapply them when images are re-posted
• identify "self-reply" threads (threads that include two or more of your own posts) and rethread them when the posts are re-posted
• identify "@" and "#" symbols that mark users' tags and hashtags within posts, and replace these symbols with similar-looking characters when re-posting so that hashtags and users are not inundated during re-posting (it is currently possible to disable this for hashtags)
• suppress "public" posts when reposting, so that they do not disrupt the local or federated timelines (it is currently possible to disable this)
• save video thumbnails
• identify posts that are bookmarked and pinned when saving, and then re-bookmark and re-pin them when they are re-posted
• selectively save or re-post only posts that are bookmarked
• selectively re-post only posts that were originally created before, after or during a specified time period

It cannot (and will never):

• set the posting date/time on re-posted content to the original posting date/time of the source post on the original Mastodon instance
• preserve or recreate threaded conversations linking your posts to those of others — this would require a feature called identity portability within Mastodon itself, and is beyond the scope of this tool
• save and restore account preferences

It cannot (and may never):

• save and restore lists
• save and restore filters

It cannot yet (and, realistically, may not for some time):

• add to an existing archive of posts on your computer (allowing you to resume saving posts if the network connection fails during that process, or to incrementally backup posts on a regular basis)
• use a Mastodon archive file as a source for post data
• save custom emojis
• re-apply downloaded video thumbnails when videos are re-posted
• quite a lot of other things (a to-do list in priority order will be linked here soon)
• repost statuses from a Mastodon instance with a longer character limit for posts to an instance with a shorter character limit for posts available from v00.01.03
• save and restore users' notes written on the profiles of other users — this may no longer be needed, as some functionality to migrate notes with users was included in Mastodon core from around mid-May, but as of 9/2 the extent of that update still needs to be confirmed

---

When should I use it?

It is best to use MastodonContentMover after you have created the account you wish to copy posts to, but before you have started migrating your followers from another account using the Mastodon migration process.

There are several reasons:

• the tool needs to be able to read and write statuses, which isn't possible when there is a redirect on the account
• when the tool reposts statuses they will appear in the timelines of those following the account, so it is best to repost a high volume of statuses before you move your followers to the new account

If you have already migrated the account you wish to archive from and your followers have all already been moved, please disable the redirect to use MastodonContentMover then reapply the redirect later (you don't need to migrate the account again, you can just reapply the redirect). If you have started migrating the account but your followers have not all been moved yet, please wait for that process to complete before disabling the redirect to use the tool. Migration usually completes within 24 hours.

---

How do I use it?

1. If necessary, download the latest release of Java from the the Oracle Java website and install it. MastodonContentMover runs within Java, similar to the way games like PUBG or Counter-Strike run on Steam.
2. Download mcm.jar, the .jar file for MastodonContentMover
3. Run mcm.jar from the command line on your computer to save your posts from your source Mastodon instance
4. Run mcm.jar again from the command line to re-post your posts to your (new) destination Mastodon instance

Important: This tool saves an access token for your Mastodon account(s) to your computer's hard drive. If someone else obtains a copy of that token, they may be able to use it (in a different tool, or in this one) to read and write posts to your Mastodon account! If you are not the only person with access to your computer, or if you are using a shared computer, please check the steps in the security section to ensure credentials are deleted after use.

Step-by-step:

1. Open a command prompt on your computer and enter "java -version".
   • If Java is installed, this will display the currently installed version.
   • If no installed version is displayed (or if the version number is less than 17.0.5), close the window then download and install Java from the official website. Then open a fresh command prompt and check again that "java -version" displays the currently installed version of Java.
2. Create a new directory for MastodonContentMover, then download mcm.jar from the project Github repository and put it in that directory.
3. Using the command prompt, move to your MastodonContentMover directory.
4. Run the following command, substituting the bold, red parameters: the domain name for your source Mastodon instance (the one you wish to save from), your username on the source instance, and the name you would like to use for the archive folder on your computer where post data will be saved. The command should be entered as one line.
   
   java -jar mcm.jar save -instance:myOldInstance.social -username:myOldUsername -archiveName:myPosts
   
   
5. Follow the prompts as needed to grant read and write permissions to MastodonContentMover using OAuth, then wait for your posts to be saved.
6. To re-post your Mastodon posts to a new instance, run the following command, substituting the bold, red parameters: the domain name for your destination Mastodon instance, your username on the new instance, and the name you used in the previous step to save your post data. The command should be entered as one line.
   
   java -jar mcm.jar post -instance:myNewInstance.world -username:myNewUsername -archiveName:myPosts
   
   
7. Follow the prompts as needed to grant read and write permissions to MastodonContentMover using OAuth, then wait for your posts to be re-posted.

Notes

Because MastodonContentMover is built using Java, the same executable file can be run on any operating system (Windows, Mac OS, Linux etc.)

Because of this, there is no installation process for the tool itself — you simply download the .jar file. Java (or the "Java Runtime Environment") also needs to be installed on your computer to run that .jar file. Java is free to download and use, and can be downloaded from the Oracle Java website.

---

Parameter guide

The main, required command-line options are as follows:

either save or post	according to whether you wish to save your existing posts from a Mastodon instance to your computer, or re-post them from a MastodonContentMover archive on your computer to a Mastodon instance
-instance	the hostname or address of the Mastodon instance
-username	your username or account name on a specified Mastodon instance
-archiveName	the name of the archive you would like to save to (create) or post from — this will be used to create or read from a folder within the data directory that will be created in your MastodonContentMover directory the first time the tool is run

Additional, optional command-line options that are currently implemented are as follows:

-bookmarkedOnly	no by default, yes to save or post only statuses that were bookmarked by the specified user account
-preserveHashtags	no by default, yes to retain hashtags when reposting statuses, which will result in the new, reposted statuses being displayed under that hashtag to anyone who follows it or searches for it providing the statuses have "public" visibility(see also: -suppressPublic)
-extraThrottle	0 by default. The number of additional seconds to wait between each API call (interaction with a given Mastodon instance)
-customPort	8081 by default, or a custom TCP port to use briefly on the local computer when you authorize the tool using OAuth in your browser
-suppressPublic	yes by default (reposts public statuses in the archive as unlisted statuses on the instance you are reposting to). Use no to retain public visiblity for public statuses in the archive (this will cause them to appear in the local and federated timelines)
-from	(only for use with post) accepts an ISO 8601 date time, to repost only those statuses in the archive with a creation date and time from this moment forward
-until	(only for use with post) accepts an ISO 8601 date time, to repost only those statuses in the archive with a creation date and time before this moment
-showDebug	no by default, yes to display additional debugging output
-testMode	no by default, yes to toggle use of idempotency key (for testing purposes only)

---

Example commands

Note: These commands may wrap onto multiple lines when shown here, but should be entered as one single line.

Saving posts from account @Gargron on https://mastodon.social to an archive called localBackup:


Re-posting saved posts from the same archive to a new account, @GargronRelocated, on https://mstdn.social:


Re-posting only the saved posts that were bookmarked:


Re-posting saved posts with public posts intact (public posts will appear in local and federated timelines):


Re-posting saved posts with hashtags intact (hashtagged posts will appear in home timelines of hashtag followers and in hashtag search results providing -suppressPublic:no is also specified):


Re-posting only saved posts with archive timestamps before midnight March 6, 2023:


Re-posting only saved posts with archive timestamps between midnight March 6, 2023, and midnight March 9, 2023:


Re-posting only saved posts with archive timestamps after midnight March 9, 2023:


Re-posting saved posts with an additional delay of 120 seconds between each API call (each request sent to the Mastodon instance):


Re-posting only the saved posts that were bookmarked and with archive timestamps between midnight March 6, 2023, and midnight March 9, 2023, with hashtags and public posts intact and an additional delay of 120 seconds between each API call:


Saving posts from account @Gargron on https://mastodon.social to an archive called localBackup with debug messages displayed:


Re-posting only the saved posts that were bookmarked and with archive timestamps between midnight March 6, 2023, and midnight March 9, 2023, with hashtags and public posts intact from the same archive to a new account, @GargronRelocated, on https://mstdn.social, with an additional delay of 120 seconds between each API call and debug messages displayed:


Saving posts from account @Gargron on https://mastodon.social to an archive called localBackup while using custom port 8080 to authenticate via OAuth (when using a custom port, credentials are not saved):


Re-posting saved posts from the same archive to a new account, @GargronRelocated, on https://mstdn.social while using custom port 8080 to authenticate via OAuth (credentials are not saved):

---

Security

This tool saves an access token for your Mastodon account(s) to your computer's hard drive. If someone else obtains a copy of that token, they may be able to use it (in a different tool, or in this one) to read and write posts to your Mastodon account!

If you are not the only person with access to your computer, or if you are using a shared computer, please delete the files that store access credentials after using this tool.

There are two files to delete, both within the root of the data directory automatically created by the tool in its working directory (the same place you saved the mcm.jar file). These files are:

• clientCredentials.xml
• userCredentials.xml

Please delete both of these files (but only these two files) after using the tool on a shared computer. You will need to authorize the tool through your browser again the next time you use it.

Adding a parameter to prevent them from being saved, as a security measure, is on the to-do list for development. In the meantime, another workaround would be to use the -customPort parameter to specify a non-default TCP port for the tool (such as 8082, for example). This will also prevent these files from being saved.

Alternatively, please consider running MastodonContentMover from a removeable drive such as a USB drive.

---

Information for instance administrators

MastodonContentMover respects Mastodon's default rate limits conservatively, allowing additional time so as not to overload instances with requests.

Purpose	Mastodon default minimum interval between calls	MastodonContentMover interval
General	1 second	5 seconds
Media uploads (per file)	60 seconds	120 seconds
New posts (non-public)	1 second	5 seconds
New posts (public)	1 second	60 seconds

Users are also encouraged to set additional throttling, using the -extraThrottle parameter, to increase the number of seconds allowed between API calls.

By default, archived posts that have "public" visibility are re-posted with "unlisted" visibility, to prevent flooding of local and federated timelines.

Here are some examples of the default throttle delays when reposting different types of statuses:

Status	MastodonContentMover interval
a status with unlisted visibility and no attached media	5 seconds
a status with unlisted visibility and one attached video	125 seconds (a 120-second media upload delay plus a 5-second non-public new post delay)
a status with public visibility in the archive but unlisted visibility when reposted (because public posts are suppressed by default), and no attached media	5 seconds
a status with public visibility in the archive and public visibility when reposted because -suppressPublic was explicitly set to no, and no attached media	60 seconds
a status with public visibility in the archive but unlisted visibility when reposted (because public posts are suppressed by default), and four attached images	485 seconds (four 120-second media upload delays plus a 5-second non-public new post delay)
a status with public visibility in the archive and public visibility when reposted because -suppressPublic was explicitly set to no, and four attached images	540 seconds (four 120-second media upload delays plus a 60-second public new post delay)

---

Further information

For now, any enquiries regarding this tool can be directed to Tokyo Outsider via Mastodon. Soon details will be made available on how to submit a problem report via Github, to report suspected bugs or difficulties with use. In the meantime, please make a note of any difficulties you encounter (ideally with a copy of the command you entered and the command line output, with -showDebug set to yes).

Because development is at an early stage and the structure of the code is likely to change with features planned for implemention, pull requests are not likely to be approved at this time, although your interest and support is appreciated. Please follow #MastodonContentMover on Mastodon for real-time development updates!

See also:

• Source code on Github
• Javadocs for the source
• The full version history

---

Acknowledgements

Creating MastodonContentMover would not have been possible without the immense efforts and tireless dedication of André Gasser and bocops, who built the BigBone Java/Kotlin library on which this tool relies (extending and updating the earlier Mastodon4J library written by Toshihiro Yagi). Their guidance, encouragement and patience has been invaluable throughout.

I'd like to bring attention to the pioneering work of Mike Macgirvin, who built seamless identity and content portability into his Zot protocol, offering an impressive model of what can technically be achieved. While stumbling blocks with ActivityPub interoperability in Hubzilla, the main platform that uses Zot, led me to return to Mastodon and build MastodonContentMover as a stopgap, the portability demonstrated is an ideal to which all Fediverse platforms will hopefully aspire.

The Mastodon team must also be thanked for their work on Mastodon itself; while their decision not to incorporate content portability into their migration mechanism (so far) has left a gap that I hope this tool will at least partially fill, they have nonetheless created the most stable, polished and functionally complete platform I've seen for short message social networking on the Fediverse.

Last but certainly not least, my enduring love, respect and gratitude to Sourabh, without whose kindness and inspiration this tool would not exist.