You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi - I was having Grok help me further develop this database application, and I don't know what happened but Grok changed something, and as far as I can tell the code looks fine, but I'm no longer getting a persistent database and cannot restore, and the log files are tiny. I'm a bit of a noob so I apologize, but at this point I would like some help with the code. Anyone willing to give me an hour of their time, I'll pay a good rate for 1 hr (100+/hr USD), we can do a screen-share on Zoom/Teams or something? It shouldn't take long.
I can show you some basic code here if you like. Environment is VS2022, C# 8.0, developing for Windows 11. This is the basic structure, seems like it's fine to me? Before exiting, it definitely has all good values held in memory...but it just never goes to disk it seems. If you can answer just from looking at this code, awesome, if not, I'm sure an hour with someone who knows better what they're doing would get me on my way. Thank you.
//defined in the class
private readonly FasterKV<byte[], byte[]> _triangleStore;
In the constructor:
// Initialize FASTER stores
var logSettings = new LogSettings
{
LogDevice = new ManagedLocalStorageDevice(Path.Combine(databasePath, "fastdb_log")),
ObjectLogDevice = new NullDevice(),
PageSizeBits = 12, // 4KB pages
MemorySizeBits = 20, // 1MB memory
SegmentSizeBits = 24 // 16MB segments
};
var checkpointSettings = new CheckpointSettings
{
CheckpointDir = Path.Combine(databasePath, "Checkpoint"),
};
ReportProgress("Initializing _triangleStore...");
_triangleStore = new FasterKV<byte[], byte[]>(1L << 20, new LogSettings
{
LogDevice = new ManagedLocalStorageDevice(Path.Combine(databasePath, "triangles_log")),
ObjectLogDevice = new NullDevice(),
PageSizeBits = logSettings.PageSizeBits,
MemorySizeBits = logSettings.MemorySizeBits,
SegmentSizeBits = logSettings.SegmentSizeBits
}, checkpointSettings);
string triangleLogPath = Path.Combine(databasePath, "triangles_log.0");
if (File.Exists(triangleLogPath))
{
ReportProgress("Recovering _triangleStore...");
_triangleStore.Recover();
ReportProgress($"Recovered {_triangleStore.EntryCount} records from _triangleStore");
}
ReportProgress($"_triangleStore initialized. Initial EntryCount: {_triangleStore.EntryCount}");
In the builder:
using var triangleSession = _triangleStore.For(new SimpleFunctions<byte[], byte[]>()).NewSession<SimpleFunctions<byte[], byte[]>>("triangle_session");
///some work
triangleSession.Upsert(ref serializedKey, ref newValue);
triangleSession.CompletePending(true);
await _triangleStore.TakeFullCheckpointAsync(CheckpointType.FoldOver);
_triangleStore.Log.Flush(true);
The database *.0 files used to be significant (MB's), now they are tiny (a few kB's), and when I try to recover, this code reports nothing is in the database, which is consistent with the .0 files being tiny:
public Dictionary<(ushort, ushort), HashSet<(ulong, ulong, ulong)>> LoadTrianglesIntoMemory(bool smallDatabase)
{
if (!smallDatabase)
{
throw new InvalidOperationException("This method is only for small test databases.");
}
var triangles = new Dictionary<(ushort, ushort), HashSet<(ulong, ulong, ulong)>>();
using var session = _triangleStore.For(new SimpleFunctions<byte[], byte[]>()).NewSession<SimpleFunctions<byte[], byte[]>>();
using var iterator = session.Iterate();
while (iterator.GetNext(out RecordInfo recordInfo))
{
byte[] key = iterator.GetKey();
if (key.Length != 4) continue; // Skip invalid keys
ushort angle1 = BitConverter.ToUInt16(key, 0);
ushort angle2 = BitConverter.ToUInt16(key, 2);
var angleKey = (angle1, angle2);
byte[] value = iterator.GetValue();
using var ms = new MemoryStream(value);
using var reader = new BinaryReader(ms);
int count = reader.ReadInt32();
var tripletSet = new HashSet<(ulong, ulong, ulong)>();
for (int i = 0; i < count; i++)
{
ulong id1 = reader.ReadUInt64();
ulong id2 = reader.ReadUInt64();
ulong id3 = reader.ReadUInt64();
tripletSet.Add((id1, id2, id3));
}
triangles[angleKey] = tripletSet;
}
ReportProgress($"Loaded {triangles.Sum(kvp => kvp.Value.Count)} triangles into memory.");
return triangles;
}
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
Hi - I was having Grok help me further develop this database application, and I don't know what happened but Grok changed something, and as far as I can tell the code looks fine, but I'm no longer getting a persistent database and cannot restore, and the log files are tiny. I'm a bit of a noob so I apologize, but at this point I would like some help with the code. Anyone willing to give me an hour of their time, I'll pay a good rate for 1 hr (100+/hr USD), we can do a screen-share on Zoom/Teams or something? It shouldn't take long.
I can show you some basic code here if you like. Environment is VS2022, C# 8.0, developing for Windows 11. This is the basic structure, seems like it's fine to me? Before exiting, it definitely has all good values held in memory...but it just never goes to disk it seems. If you can answer just from looking at this code, awesome, if not, I'm sure an hour with someone who knows better what they're doing would get me on my way. Thank you.
In the constructor:
In the builder:
The database *.0 files used to be significant (MB's), now they are tiny (a few kB's), and when I try to recover, this code reports nothing is in the database, which is consistent with the .0 files being tiny:
Beta Was this translation helpful? Give feedback.
All reactions