Skip to content

Add file-based persistent log store and state manager #635

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bihari123 wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add file-based persistent log store and state manager #635

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions examples/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
add_executable(calc_server
calculator/calc_server.cxx
logger.cc
in_memory_log_store.cxx)
in_memory_log_store.cxx
file_based_log_store.cxx)

# add_dependencies(calc_server
# ${LIBRARY_NAME})
Expand All @@ -14,7 +15,8 @@ target_link_libraries(calc_server
add_executable(echo_server
echo/echo_server.cxx
logger.cc
in_memory_log_store.cxx)
in_memory_log_store.cxx
file_based_log_store.cxx)
Copy link
Copy Markdown
Contributor

@greensky00 greensky00 Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

echo_server will not use it, after it is removed from example_common.hxx.

# add_dependencies(echo_server
# ${LIBRARY_NAME})
target_link_libraries(echo_server
Expand All @@ -24,7 +26,8 @@ target_link_libraries(echo_server
add_executable(quick_start
quick_start.cxx
logger.cc
in_memory_log_store.cxx)
in_memory_log_store.cxx
file_based_log_store.cxx)
Copy link
Copy Markdown
Contributor

@greensky00 greensky00 Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

# add_dependencies(quick_start
# ${LIBRARY_NAME})
target_link_libraries(quick_start
Expand Down
13 changes: 11 additions & 2 deletions examples/calculator/calc_server.cxx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ limitations under the License.

#include "calc_state_machine.hxx"
#include "in_memory_state_mgr.hxx"
#include "file_based_state_mgr.hxx"
#include "logger_wrapper.hxx"

#include "nuraft.hxx"
Expand Down Expand Up @@ -220,6 +221,11 @@ void check_additional_flags(int argc, char** argv) {
CALL_TYPE = raft_params::async_handler;
} else if (strcmp(argv[ii], "--async-snapshot-creation") == 0) {
ASYNC_SNAPSHOT_CREATION = true;
} else if (strcmp(argv[ii], "--persistent") == 0) {
set_persistent_storage(true);
} else if (strncmp(argv[ii], "--persistent-dir=", 17) == 0) {
std::string dir = argv[ii] + 17;
set_persistent_storage(true, dir);
}
}
}
Expand All @@ -231,8 +237,11 @@ void calc_usage(int argc, char** argv) {
ss << std::endl << std::endl;
ss << " options:" << std::endl;
ss << " --async-handler: use async type handler." << std::endl;
ss << " --async-snapshot-creation: create snapshots asynchronously."
<< std::endl << std::endl;
ss << " --async-snapshot-creation: create snapshots asynchronously." << std::endl;
ss << " --persistent: enable file-based persistent storage." << std::endl;
ss << " --persistent-dir=<dir>: specify directory for persistent storage" << std::endl;
ss << " (default: ./nuraft_data)." << std::endl
<< std::endl;

std::cout << ss.str();
exit(0);
Expand Down
29 changes: 26 additions & 3 deletions examples/example_common.hxx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,21 @@ using namespace nuraft;

using raft_result = cmd_result< ptr<buffer> >;

// Global flag to enable persistent storage.
// If true, file-based storage will be used instead of in-memory.
static bool USE_PERSISTENT_STORAGE = false;

// Directory for persistent storage.
static std::string PERSISTENT_STORAGE_DIR = "./nuraft_data";

// Helper to set persistent storage options.
inline void set_persistent_storage(bool enable, const std::string& dir = "./nuraft_data") {
USE_PERSISTENT_STORAGE = enable;
if (enable) {
PERSISTENT_STORAGE_DIR = dir;
}
}

Comment on lines +24 to +38
Copy link
Copy Markdown
Contributor

@greensky00 greensky00 Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are used by calc_server only, not common features. Need to move it to calc_server.cxx.

Copy link
Copy Markdown
Contributor

@greensky00 greensky00 Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+)
After moving them to calc_server, set default dir path to ./calc_server_data_<server_id>.

struct server_stuff {
server_stuff()
: server_id_(1)
Expand Down Expand Up @@ -155,10 +170,18 @@ void init_raft(ptr<state_machine> sm_instance) {
ptr<logger_wrapper> log_wrap = cs_new<logger_wrapper>( log_file_name, 4 );
stuff.raft_logger_ = log_wrap;

// State manager: use persistent or in-memory based on flag.
if (USE_PERSISTENT_STORAGE) {
std::cout << "using persistent storage: " << PERSISTENT_STORAGE_DIR << std::endl;
stuff.smgr_ = cs_new<file_based_state_mgr>( stuff.server_id_,
stuff.endpoint_,
PERSISTENT_STORAGE_DIR );
} else {
std::cout << "using in-memory storage" << std::endl;
stuff.smgr_ = cs_new<inmem_state_mgr>( stuff.server_id_,
stuff.endpoint_ );
}
Comment on lines +173 to +183
Copy link
Copy Markdown
Contributor

@greensky00 greensky00 Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, I suggest modifying init_raft function to take an additional parameter ptr<state_mgr>, which is inmem_state_mgr by default. Only calc_server would pass file-based state mgr.

// State machine.
stuff.smgr_ = cs_new<inmem_state_mgr>( stuff.server_id_,
stuff.endpoint_ );
// State manager.
stuff.sm_ = sm_instance;

// ASIO options.
Expand Down
Loading
Loading