Downgrading MongoDB 8.0 to 7.0

# downgrade
Jean da Silva
Published date:
8 min read

On newer releases of MongoDB, more precisely after 7.0, the downgrade action, which used to be a straightforward action with a documented process, became a big interrogation point after Mongo Inc. released the following notes: Downgrade 7.0 to 6.0

  The downgrade procedure got even less verbose with the arrival of MongoDB 8.0, which says only the following: Downgrade 8.0 to 7.0

No other observation was mentioned in the release notes. In a quick summary:

  At this point, you might have wondered:

“Is, is the downgrade process still possible?”

The short answer is Yes, but with observations as I will share below. To get started here, let’s understand how the process works on MongoDB 8.0.

rs0 [direct: primary] admin> db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )
{
  featureCompatibilityVersion: { version: '8.0' },
  ok: 1,
  '$clusterTime': {
    clusterTime: Timestamp({ t: 1739388689, i: 1 }),
    signature: {
      hash: Binary.createFromBase64('Ufa9JQOJi8bbBFs0VPuQw1wZhi4=', 0),
      keyId: Long('7470593229067386887')
    }
  },
  operationTime: Timestamp({ t: 1739388689, i: 1 })
}

As documented here. Either on upgrade or downgrade, MongoDB follows 4 distinct phases to change the version, which are all handled by the FCV.

  That being said, let’s go through the internal phases that happen during the downgrade process on a Replica Set.

Phase 1: Transition initiation:

The first phase is about signaling intent to the entire cluster that a downgrade is beginning. This is handled by the setFeatureCompatibilityVersion command:

db.adminCommand({setFeatureCompatibilityVersion: "7.0", confirm: true})

Once that is issued, Mongo starts transitioning to requestedVersion by updating the local FCV document tokDowngradingFrom_X_To_Y state, respectively. This step is expected to be fast and always succeed. Well, mongo requires an exclusive lock for that operation and ensures no other FCV change is happening:

// Only allow one instance of setFeatureCompatibilityVersion to run at a time.
Lock::ExclusiveLock setFCVCommandLock(opCtx, commandMutex);

Watching that transition phase in the mongo.log, you should see something as below:

{"t":{"$date":"2025-07-08T14:24:47.608-03:00"},"s":"I",  "c":"-",        "id":6744300, "ctx":"conn47","msg":"setFeatureCompatibilityVersion command called","attr":{"upgradeOrDowngrade":"downgrade","serverType":"replica set/maintenance mode","fromVersion":"8.0","toVersion":"7.0"}}
[...]
{"t":{"$date":"2025-07-08T14:24:47.608-03:00"},"s":"I", "c":"REPL", "id":20459, "ctx":"conn47","msg":"Setting featureCompatibilityVersion","attr":{"currentVersion":"8.0","newVersion":"downgrading from 8.0 to 7.0"}} 
[...]
{"t":{"$date":"2025-07-08T14:24:47.618-03:00"},"s":"I", "c":"-", "id":6744301, "ctx":"conn47","msg":"setFeatureCompatibilityVersion has set the FCV to the transitional state","attr":{"upgradeOrDowngrade":"downgrade","serverType":"replica set/maintenance mode","fromVersion":"8.0","toVersion":"7.0"}}

Not only that, but worth mentioning that in this phase, Mongo ensures a few other things, such as aborting all running index builders that might be incompatible, and updates internal wire protocol specifications

Phase 2: Validation and Preparation:

In this second phase, the downgrade-ability checks are performed via _prepareToDowngrade and its sub-functions as  _prepareToDowngradeActions. The objective here is to be a safety validation phase, to ensure the downgrade can proceed safely before making any metadata changes, and no 8.0-only features are in use before proceeding with the actual downgrade operations. On this part, NO metadata cleanup is performed yet.

_prepareToUpgradeActions(opCtx, requestedVersion, changeTimestamp);

        {
            // Take the global lock in S mode to create a barrier for operations taking the global
            // IX or X locks. This ensures that either:
            //   - The global IX/X locked operation will start after the FCV change, see the
            //     upgrading to the latest FCV and act accordingly.
            //   - The global IX/X locked operation began prior to the FCV change, is acting on that
            //     assumption and will finish before upgrade procedures begin right after this.
            Lock::GlobalLock lk(opCtx, MODE_S);
        }
uassert(ErrorCodes::Error(549181),
                "Failing downgrade due to 'failDowngrading' failpoint set",
                !failDowngrading.shouldFail());
        hangWhileDowngrading.pauseWhileSet(opCtx);

        // This helper function is for any uasserts for users to clean up user collections. Uasserts
        // for users to change settings or wait for settings to change should also happen here.
        // These uasserts happen before the internal server downgrade cleanup. The code in this
        // helper function is required to be idempotent in case the node crashes or downgrade fails
        // in a way that the user has to run setFCV again. The code added/modified in this helper
        // function should not leave the server in an inconsistent state if the actions in this
        // function failed part way through.
        // This helper function can only fail with some transient error that can be retried (like
        // InterruptedDueToReplStateChange) or ErrorCode::CannotDowngrade. The uasserts added to
        // this helper function can only have the CannotDowngrade error code indicating that the
        // user must manually clean up some user data in order to retry the FCV downgrade.
        _userCollectionsUassertsForDowngrade(opCtx, requestedVersion);

This is the phase where MongoDB will complain if it identifies that your cluster is using an 8.0-specific feature that cannot be used on 7.0 That’s why the best action before any downgrade procedure is to review the current version and analyze whether the environment is using any of those new features.

Moving on, and to demonstrate how mongo would fail on the prepare step, we enabled featureFlagRecordIdsReplicated:

/mongod --replSet replset --dbpath /replset/rs1/db --logpath /replset/rs1/mongod.log --port 8083 --fork --keyFile /keyfile --wiredTigerCacheSizeGB 1 --setParameter featureFlagRecordIdsReplicated=true

This MongoDB 8.0 feature ensures RecordId consistency across replica set members. RecordIds are internal storage engine identifiers and with recordIdsReplicated enabled, the primary node includes RecordIds in oplog entries (as ‘rid’ fields), forcing secondary nodes to use identical RecordIds for the same documents. This creates a replication protocol change that MongoDB 7.0 cannot understand, making it incompatible for downgrades. For example:

replset [direct: primary] blogpost> db.createCollection("test_recordids", {
    recordIdsReplicated: true
})

replset [direct: primary] blogpost> db.test_recordids.insertMany([
...     { name: "Alice", age: 25, city: "New York" },
...     { name: "Bob", age: 30, city: "London" },
...     { name: "Charlie", age: 35, city: "Tokyo" }
... ])
{
  acknowledged: true,
  insertedIds: {
    '0': ObjectId('686ea382fd1484cd608b1fbd'),
    '1': ObjectId('686ea382fd1484cd608b1fbe'),
    '2': ObjectId('686ea382fd1484cd608b1fbf')
  }
}
replset [direct: primary] local> db.oplog.rs.find({"ns": "blogpost.test_recordids","op": "i"},{'o._id':1 ,rid: 1})
[
  { o: { _id: ObjectId('686ea382fd1484cd608b1fbd') }, rid: Long('1') },
  { o: { _id: ObjectId('686ea382fd1484cd608b1fbe') }, rid: Long('2') },
  { o: { _id: ObjectId('686ea382fd1484cd608b1fbf') }, rid: Long('3') }
]

-- SECODARY -- 
use local 
replset [direct: secondary] local> db.oplog.rs.find({"ns": "blogpost.test_recordids","op": "i"},{'o._id':1 ,rid: 1})
[
  { o: { _id: ObjectId('686ea382fd1484cd608b1fbd') }, rid: Long('1') },
  { o: { _id: ObjectId('686ea382fd1484cd608b1fbe') }, rid: Long('2') },
  { o: { _id: ObjectId('686ea382fd1484cd608b1fbf') }, rid: Long('3') }
]

As we can see, each document was replicated, carrying over the rid they got from PRIMARY. On the other hand, in a regular collection, the replication looks as follows:

replset [direct: primary] blogpost> db.test_regular.insertMany([
...     { name: "Alice", age: 25, city: "New York" },
...     { name: "Bob", age: 30, city: "London" },
...     { name: "Charlie", age: 35, city: "Tokyo" }
... ])
{
  acknowledged: true,
  insertedIds: {
    '0': ObjectId('686ea5ad15ad32da8c8b1fbd'),
    '1': ObjectId('686ea5ad15ad32da8c8b1fbe'),
    '2': ObjectId('686ea5ad15ad32da8c8b1fbf')
  }
}

replset [direct: primary] local> db.oplog.rs.find({"ns": "blogpost.test_regular","op": "i"},{'o._id':1 ,rid: 1})
[
  { o: { _id: ObjectId('686ea5ad15ad32da8c8b1fbd') } },
  { o: { _id: ObjectId('686ea5ad15ad32da8c8b1fbe') } },
  { o: { _id: ObjectId('686ea5ad15ad32da8c8b1fbf') } }
]

No rid was used . If we try to downgrade, we will face the message below during the prepare phase:

replset [direct: primary] blogpost> db.adminCommand( { setFeatureCompatibilityVersion: "7.0", confirm: true } )
MongoServerError[CannotDowngrade]: Cannot downgrade the cluster when there are collections with 'recordIdsReplicated' enabled. Please unset the option or drop the collection(s) before downgrading. First detected collection with 'recordIdsReplicated' enabled: blogpost.test_recordids (UUID: 0ed7067b-8384-4e08-b66e-f0a781405ea6).
{"t":{"$date":"2025-07-09T02:06:40.869-03:00"},"s":"I",  "c":"-",        "id":6744301, "ctx":"conn42","msg":"setFeatureCompatibilityVersion has set the FCV to the transitional state","attr":{"upgradeOrDowngrade":"downgrade","serverType":"replica set/maintenance mode","fromVersion":"8.0","toVersion":"7.0"}}
{"t":{"$date":"2025-07-09T02:06:40.870-03:00"},"s":"D1", "c":"ASSERT",   "id":23074,   "ctx":"conn42","msg":"User assertion","attr":{"error":"CannotDowngrade: Cannot downgrade the cluster when there are collections with 'recordIdsReplicated' enabled. Please unset the option or drop the collection(s) before downgrading. First detected collection with 'recordIdsReplicated' enabled: blogpost.test_recordids (UUID: 0ed7067b-8384-4e08-b66e-f0a781405ea6).","file":"src/mongo/db/commands/set_feature_compatibility_version_command.cpp","line":1265}}
{"t":{"$date":"2025-07-09T02:06:40.870-03:00"},"s":"D3", "c":"ASSERT",   "id":4892201, "ctx":"conn42","msg":"Internal assertion","attr":{"error":{"code":332,"codeName":"CannotDowngrade","errmsg":"Cannot downgrade the cluster when there are collections with 'recordIdsReplicated' enabled. Please unset the option or drop the collection(s) before downgrading. First detected collection with 'recordIdsReplicated' enabled: blogpost.test_recordids (UUID: 0ed7067b-8384-4e08-b66e-f0a781405ea6)."},"location":"{fileName:\"src/mongo/db/service_entry_point_common.cpp\", line:1449, functionName:\"_handleError\"}"}}
{"t":{"$date":"2025-07-09T02:06:40.870-03:00"},"s":"D3", "c":"ASSERT",   "id":4892201, "ctx":"conn42","msg":"Internal assertion","attr":{"error":{"code":332,"codeName":"CannotDowngrade","errmsg":"Cannot downgrade the cluster when there are collections with 'recordIdsReplicated' enabled. Please unset the option or drop the collection(s) before downgrading. First detected collection with 'recordIdsReplicated' enabled: blogpost.test_recordids (UUID: 0ed7067b-8384-4e08-b66e-f0a781405ea6)."},"location":"{fileName:\"src/mongo/db/service_entry_point_common.cpp\", line:730, functionName:\"run\"}"}}
{"t":{"$date":"2025-07-09T02:06:40.870-03:00"},"s":"D1", "c":"COMMAND",  "id":21962,   "ctx":"conn42","msg":"Assertion while executing command","attr":{"command":"setFeatureCompatibilityVersion","db":"admin","commandArgs":{"setFeatureCompatibilityVersion":"7.0","confirm":true,"lsid":{"id":{"$uuid":"88d019cc-8649-4963-9e7c-1297c79d45de"}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1752037588,"i":1}},"signature":{"hash":{"$binary":{"base64":"rK0IXY19pYAiE/3DaS3Rw4rRhm8=","subType":"0"}},"keyId":7519659317708980230}},"$readPreference":{"mode":"primaryPreferred"},"$db":"admin"},"error":"CannotDowngrade: Cannot downgrade the cluster when there are collections with 'recordIdsReplicated' enabled. Please unset the option or drop the collection(s) before downgrading. First detected collection with 'recordIdsReplicated' enabled: blogpost.test_recordids (UUID: 0ed7067b-8384-4e08-b66e-f0a781405ea6)."}}
{"t":{"$date":"2025-07-09T02:06:40.870-03:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn42","msg":"Slow query","attr":{"type":"command","isFromUserConnection":true,"ns":"admin.$cmd","collectionType":"admin","appName":"mongosh 2.5.3","command":{"setFeatureCompatibilityVersion":"7.0","confirm":true,"lsid":{"id":{"$uuid":"88d019cc-8649-4963-9e7c-1297c79d45de"}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1752037588,"i":1}},"signature":{"hash":{"$binary":{"base64":"rK0IXY19pYAiE/3DaS3Rw4rRhm8=","subType":"0"}},"keyId":7519659317708980230}},"$readPreference":{"mode":"primaryPreferred"},"$db":"admin"},"numYields":0,"ok":0,"errMsg":"Cannot downgrade the cluster when there are collections with 'recordIdsReplicated' enabled. Please unset the option or drop the collection(s) before downgrading. First detected collection with 'recordIdsReplicated' enabled: blogpost.test_recordids (UUID: 0ed7067b-8384-4e08-b66e-f0a781405ea6).","errName":"CannotDowngrade","errCode":332,"reslen":509,"locks":{"MultiDocumentTransactionsBarrier":{"acquireCount":{"R":1}},"ReplicationStateTransition":{"acquireCount":{"w":10}},"Global":{"acquireCount":{"r":8,"w":3,"R":1}},"Database":{"acquireCount":{"r":6,"w":3}},"Collection":{"acquireCount":{"r":4,"w":3}},"Mutex":{"acquireCount":{"W":2}}},"flowControl":{"acquireCount":3},"readConcern":{"level":"local","provenance":"implicitDefault"},"writeConcern":{"w":"majority","wtimeout":0,"provenance":"implicitDefault"},"storage":{},"cpuNanos":2236818,"remote":"127.0.0.1:60690","protocol":"op_msg","queues":{"execution":{"admissions":11},"ingress":{"admissions":2}},"workingMillis":7,"durationMillis":7}}

After successfully passing the validation checks in Phase 2, MongoDB moves into Phase 3. The core cleanup phase, where the database performs the actual internal server metadata changes required for the downgrade. 

It’s in this phase that MongoDB transitions from validation to action. Having confirmed that the downgrade is safe to proceed, the database now begins the process of removing 8.0-specific server metadata and preparing the internal systems to operate under MongoDB 7.0 constraints.

// Set the isCleaningServerMetadata field to true. This prohibits the downgrading to
// upgrading transition until the isCleaningServerMetadata is unset when we successfully
// finish the FCV downgrade and transition to the DOWNGRADED state.
{
const auto fcvChangeRegion(FeatureCompatibilityVersion::enterFCVChangeRegion(opCtx));
FeatureCompatibilityVersion::updateFeatureCompatibilityVersionDocument(
opCtx,
actualVersion,
requestedVersion,
isFromConfigServer,
changeTimestamp,
true /* setTargetVersion */,
true /* setIsCleaningServerMetadata*/);
}

This flag serves as a safety mechanism that prevents the cluster from accidentally switching back to an upgrade operation while cleanup is in progress. Think of it as a “Do Not Disturb” sign that ensures the downgrade process completes atomically. 

Another core component of this phase is the _internalServerCleanupForDowngrade function:

// This helper function is for any internal server downgrade cleanup, such as dropping
// collections or aborting. This cleanup will happen after user collection downgrade
// cleanup.
// The code in this helper function is required to be IDEMPOTENT and RETRYABLE in case the
// node crashes or downgrade fails in a way that the user has to run setFCV again. It cannot
// fail for a non-retryable reason since at this point user data has already been cleaned
// up.
// It also MUST be able to be rolled back. This is because we cannot guarantee the safety of
// any server metadata that is not replicated in the event of a rollback.
//
// This helper function can only fail with some transient error that can be retried
// (like InterruptedDueToReplStateChange), ManualInterventionRequired, or fasserts. For
// any non-retryable error in this helper function, it should error either with an
// uassert with ManualInterventionRequired as the error code (indicating a server bug
// but that all the data is consistent on disk and for reads/writes) or with an fassert
// (indicating a server bug and that the data is corrupted). ManualInterventionRequired
// and fasserts are errors that are not expected to occur in practice, but if they did,
// they would turn into a Support case.
_internalServerCleanupForDowngrade(
opCtx, getTransitionFCVInfo(fcvSnapshot.getVersion()).from, requestedVersion);

In short, that  cleanup function has three critical requirements:

  1. IDEMPOTENT: Can be run multiple times safely
  2. RETRYABLE: Can recover from transient failures
  3. ROLLBACK SAFE: Can be undone if necessary

In the MongoDB logs, Phase 3 activities would appear as:

{"t":{"$date":"2025-07-09T15:00:42.458-03:00"},"s":"I", "c":"WRITE", "id":51803, "ctx":"conn616","msg":"Slow query","attr":{"type":"update","isFromUserConnection":true,"ns":"admin.system.version","collectionType":"system","appName":"mongosh 2.5.3","command":{"q":{"_id":"featureCompatibilityVersion"},"u":{"_id":"featureCompatibilityVersion","version":"7.0","targetVersion":"7.0","previousVersion":"8.0","isCleaningServerMetadata":true},"multi":false,"upsert":true},"planSummary":"IDHACK","planningTimeMicros":26,"totalOplogSlotDurationMicros":103,"keysExamined":1,"docsExamined":1,"nMatched":1,"nModified":1,"nUpserted":0,"keysInserted":0,"keysDeleted":0,"numYields":0,"locks":{"MultiDocumentTransactionsBarrier":{"acquireCount":{"R":1}},"ReplicationStateTransition":{"acquireCount":{"w":18}},"Global":{"acquireCount":{"r":15,"w":4,"R":1}},"Database":{"acquireCount":{"r":11,"w":4}},"Collection":{"acquireCount":{"r":25,"w":4}},"Mutex":{"acquireCount":{"W":3}},"oplog":{"acquireCount":{"r":1}}},"flowControl":{"acquireCount":4},"readConcern":{"level":"local","provenance":"implicitDefault"},"storage":{"data":{"txnBytesDirty":889}},"cpuNanos":403960,"remote":"127.0.0.1:47132","queues":{"execution":{"admissions":19},"ingress":{"admissions":3}},"workingMillis":0,"durationMillis":0}}

The FCV document was updated to include isCleaningServerMetadata:true and MongoDB performed cluster parameter cleanup operations

This phase doesn’t have a separate _finalizeDowngrade function, unlike upgrades, which include a dedicated _finalizeUpgrade step. Downgrades complete with a single, atomic document update, reflecting a fundamental difference in the nature of upgrades versus downgrades.

That single, atomic operation transforms the FCV document from its transitional state to the final target version:

{
  "_id": "featureCompatibilityVersion",
  "version": "7.0",
  "targetVersion": "7.0", 
  "previousVersion": "8.0",
  "isCleaningServerMetadata": true
}
{
  "_id": "featureCompatibilityVersion",
  "version": "7.0"
}

Looking at the MongoDB source code, the core operation implemented in the SetFeatureCompatibilityVersionCommand:

// Complete transition by updating the local FCV document to the fully upgraded or
            // downgraded requestedVersion.
            const auto fcvChangeRegion(FeatureCompatibilityVersion::enterFCVChangeRegion(opCtx));

            uassert(ErrorCodes::Error(6794601),
                    "Failing downgrade due to 'failBeforeUpdatingFcvDoc' failpoint set",
                    !failBeforeUpdatingFcvDoc.shouldFail());

            hangBeforeUpdatingFcvDoc.pauseWhileSet();

            FeatureCompatibilityVersion::updateFeatureCompatibilityVersionDocument(
                opCtx,
                serverGlobalParams.featureCompatibility.acquireFCVSnapshot().getVersion(),
                requestedVersion,
                isFromConfigServer,
                changeTimestamp,
                false /* setTargetVersion */,
                false /* setIsCleaningServerMetadata */);
        }

From the mongo.log, if this phase succeeds, you should see log entry like below:

{"t":{"$date":"2025-07-09T02:00:11.129-03:00"},"s":"I", "c":"-", "id":6744302, "ctx":"conn50","msg":"setFeatureCompatibilityVersion succeeded","attr":{"upgradeOrDowngrade":"downgrade","serverType":"replica set/maintenance mode","fromVersion":"8.0","toVersion":"7.0"}}

How about the binaries downgrade?

That’s where it got a bit nebulous. While the FCV mechanism is very well detailed in the code and internal docs and has some details in the main doc as well, binary downgrade is the opposite. Barely to minimum is mentioned other than digging into the code, which is also very silent on that process. The core difference is architectural. FCV is a developed internal mechanism with clear phases and validation; all phases are catered to handle it as well as possible. Binary change(upgrade/downgrade) involves a trial-and-error attempt. As soon as you replace the binaries, even before changing the FCV, file verification happens automatically through a compatibility check to ensure those files can be read:

void WiredTigerKVEngine::_openWiredTiger(const std::string& path, const std::string& wtOpenConfig) {
    // MongoDB 4.4 will always run in compatibility version 10.0.
    std::string configStr = wtOpenConfig + ",compatibility=(require_min=\"10.0.0\")";
    auto wtEventHandler = _eventHandler.getWtEventHandler();

    int ret = wiredtiger_open(path.c_str(), wtEventHandler, configStr.c_str(), &_conn);
    if (!ret) {
        _fileVersion = {WiredTigerFileVersion::StartupVersion::IS_44_FCV_44};
        return;
    }

    if (_eventHandler.isWtIncompatible()) {
        // WT 4.4+ will refuse to startup on datafiles left behind by 4.0 and earlier. This behavior
        // is enforced outside of `require_min`. This condition is detected via a specific error
        // message from WiredTiger.
        if (_inRepairMode) {
            // In case this process was started with `--repair`, remove the "repair incomplete"
            // file.
            StorageRepairObserver::get(getGlobalServiceContext())->onRepairDone(nullptr);
        }
        LOGV2_FATAL_NOTRACE(
            4671205,
            "This version of MongoDB is too recent to start up on the existing data files. "
            "Try MongoDB 4.2 or earlier.");
    }
[...]

Well, the truth is that the trial-and-error approach is always used when you start up your instance, regardless of the version. It’s mongo validating if the data files are compatible with that binary version.

Conclusion.

It’s important to clarify that nothing blocks you from downgrading your environment. Even though the downgrade procedure no longer exists, you can still reference the old ones, such as 6.0 to 5.0. That said, while MongoDB’s documentation may discourage binary downgrades for Community Edition users, the underlying FCV mechanism provides a technically sound foundation for version transitions, and that’s the component that performs the heavy lifting when changing between versions. It’s interesting to see how robust that process is. At this point, you have two very clear paths to follow when planning your downgrade process.

Alongside that, there are 2 major points that can influence your final decision: whether or not you have a support contract for your environment.

With a solid backup/restore plan in place, proper testing, and a well-structured Replica Set/Shard Cluster, you are severely mitigating the risks, even those that a binary change can have; the key is understanding the process, respecting the safety mechanisms, and thoroughly testing any downgrade procedures in your specific environment.   Let me know your thoughts on this content in the comment section below. See ya!

Share
Next
Benchmarking MongoDB from 3.6 to 8.0 - Part 2