(READY FOR REVIEW)Garbage collect: A stop-the-world cycle collector #4180

discord9 · 2022-09-22T14:56:42Z

That might be extend to be a on-the-fly version. Based on #4158

Summary of GC Algorithm

the algorithm store a buffer of object thats' "possible roots" of garbage cycles(i.e. if a object is decrement but not to zero, its a possible root) then, during gc, algorithm have three main phase:

iterate over this buffer, free objects of rc==0.
then in next phase tracing other object(whose rc>0) and do a "trial decrement" to all of its childrens, so to remove all the "internal reference" in the graph, so whats left is external reference, so by now,object with rc==0 is safe to be collected
restore everyone elses rc, only drop garbage found from previous phase, note in here we first do a drop in place for every garbage then dealloc everything to prevent UB from deref to deallocated objects

Progress

future features for GC

use MemBalancer to determine when to gc(actually tried on a new branch, but I am having difficulty sampling memory usage&memory deallocate rate so not having a very good effect)
found out how to make optional trait bound without using typeid&macro(which make it hard to extend)

GC condition

gc will be triggered if the internal buffer(roots) have exceed 10007(Just a random reasonably large number) element and at least 100ms after last gc, that means after 10007 different objects get decref after last gc, a new gc will be trigger, gc() is only called in the loop of executing instruction, but also reexport a gc() to stdlib so one can call gc() from Python side(which will force a gc regardless of above conditon).
stop-the-world is impl by try to blocking on a pausing Mutex owned by gc whenever try to deref PyObjectRef/PyRef/Py

Basic ideas

cycle collector algorithm:

stop the world, and remove internal reference
found out those with zero ref(meaning they are cyclic garbage), mark them for later free, restore others' ref count.
free those marked object(also unpause the world), also for those marked object form a cycle, some check is done to make sure no double drop or deallocate happen

Data struct

#[repr(C)]
struct PyInner<T> {
    #[cfg(not(feature = "gc"))]
    ref_count: RefCount,
    #[cfg(feature = "gc")]
    header: GcHeader,
    ....

/// Garbage collect header, containing ref count and other info, using repr(C) to stay consistent with PyInner 's repr
#[repr(C)]
pub struct GcHeader {
    ref_cnt: PyAtomic<usize>,
    color: PyMutex<Color>,
    buffered: PyMutex<bool>,
    pub exclusive: PyMutex<()>,
    pub gc: PyRc<CcSync>
}

Example

for code as below:

tot = 0
runs = 16000
for i in range(runs):
    a = [0]
    b = [a]
    a[0] = b
    del a
    del b
    a = {}
    b = {}
    a[0] = b
    b[0] = a
    del a
    del b

With and without gc, the memory usage is(run on a win10(19044.2006) with rustc 1.64.0, might vary between different runs):

loop	mem with gc(MB)	without gc(MB)
0	13.29	8.82
1000	13.49	10.36
2000	13.76	11.76
3000	14.29	13.32
4000	14.29	14.45
5000	14.31	16.32
6000	14.31	17.24
7000	14.31	18.32
8000	14.31	19.45
9000	15.36	21.38
10000	15.36	22.45
11000	15.26	23.71
12000	15.26	24.63
13000	15.26	25.97
14000	15.26	27.36
15000	15.26	28.36

currently `GcTrace`d types are:

// builtin types
// PyRange, PyStr is acyclic
PyBoundMethod,
PyDict,
PyEnumerate,
PyFilter,
PyFunction,
PyList,
PyMappingProxy,
PyProperty,
PySet,
PySlice,
PyStaticMethod,
PySuper,
PyTraceback,
PyTuple,
// PyType, (strange bug, see PyType's trace() for detail)
PyZip,
// misc
PyCell,
// iter in iter.rs
PySequenceIterator,
PyCallableIterator,
// iter on types
// PyList's iter
PyListIterator,
PyListReverseIterator,
// PyTuple's iter
PyTupleIterator,
// PyEnumerate's iter
PyReverseSequenceIterator,
// PyMemory's iter
PyMemoryViewIterator,
// function/Arg protocol
ArgCallable,
ArgIterable,
ArgMapping,
ArgSequence,
// protocol
// struct like
PyBuffer,
PyIter,
// FIXME(discord9): confirm this is ok to do
PyIterIter<T>,
PyIterReturn,
PyMapping,
PyNumber,
PySequence

discord9 · 2022-09-23T01:23:40Z

A simple example of collecting cycle ref of PyList(note the line of collect cyclic garbage composed of two object(a and b in this case)):

a= [0]
b = [a]
a[0] = b

fanninpm · 2022-09-23T03:13:22Z

Rust 1.64 came out a few hours ago, so there are some new compilation warnings and Clippy lints to deal with in a separate PR (#4182). Once that PR is merged, it might be a good idea to rebase this branch on top of a fresh copy of main.

vm/src/object/gc/trace.rs

discord9 · 2022-09-25T13:43:19Z

Just realize I still need make some change to make gc adopt to threading, need to think about it.
edited: I am using a mutex to stop the world, which have serveal performance issue, considering changing to PyRwLock instead, too much todo, change to draft for now
edited1: done

discord9 · 2022-09-26T16:51:36Z

bug: might need to store the number of PyWeak ref count to prevent premature dealloc in heap and access violation. Also PyWeak itself is storing a PyObjectRef which keep the object from drop, some change need to be made to change that
edited: still haven't fix those failed test case now
edited1: done (basically, see progress section

fanninpm · 2022-09-27T15:33:54Z

gc will be triggered if the internal buffer(roots) have exceed 100 element(could come up with better condition in the future)

CPython has a generational garbage collector with a total of three generations. The first generation has a threshold of 700 objects, and the next two have a threshold of 10 objects each. ¹ I imagine that bumping the current threshold from 100 up to 700 objects may mitigate some of the performance decrease you're currently observing.

https://devguide.python.org/internals/garbage-collector/index.html#optimization-generations ↩

discord9 · 2022-10-08T11:57:46Z

Sorry for the messy commit logs, will reword them later.
edited: sort into six commits for the ease of code review

fanninpm · 2022-10-11T14:35:27Z

When happens when you compile this with the --no-default-features flag? I suspect there may be a few compilation errors with stdlib/src/gc.rs.

discord9 · 2022-10-12T01:02:11Z

Yes, just realize that, gonna fix it, also still debugging on the dead lock&double dealloc issue, my code is still very faulty
edited: done, basically limit trace to those object that is GcTrace and dead lock disappear.

youknowone · 2022-10-16T14:49:15Z

@coolreader18 do you have advices?

discord9 · 2022-10-22T09:25:12Z

Might found a bug related to the old ref count system: #4238 ,still trying to find a fix
Edited: solved

discord9 · 2022-11-07T03:44:34Z

Using PyRwLock is indeed very slow, might need to found out a way to trace PyMutex properly

DimitrisJim · 2022-11-17T21:38:25Z

ED48

I've slightly taken a peak at the paper this is based on, I'm guessing the two improvements it mentions have been implemented, right? I.e, using the buffered flag to not re-add it to the collector list and checking the current items in the collector as a single graph and not as distinct components.

discord9 · 2022-11-18T03:06:34Z

I've slightly taken a peak at the paper this is based on, I'm guessing the two improvements it mentions have been implemented, right? I.e, using the buffered flag to not re-add it to the collector list and checking the current items in the collector as a single graph and not as distinct components.

Yes, this is the main improvement here, I write a buffered field for every object and there is a roots vec, actually the name of methods is corresponding between paper and the actual code

coolreader18

Thank you so much for working on this! I started this a week or 2 ago and then it slipped my mind after I took a break about halfway through looking at stuff. I haven't looked at the actual gc code, and though I'd really like to, to be honest I'm not sure I'll have the bandwidth to do so. If no other maintainers feel able to either, worst case we mark it as a sort of experimental feature, but the test suite passing fine gives me confidence that it's working more or less as it should. It's not like there haven't already been multithreaded gc issues even without the tracing gc, lol.

coolreader18 · 2022-11-21T04:24:46Z

stdlib/src/gc.rs

-        Err(vm.new_not_implemented_error("".to_owned()))
+        #[cfg(feature = "gc")]
+        {
+            rustpython_vm::object::disable();


It would be good for these to have more descriptive names, either a gc_ prefix or a separate module. Also the _gc module could probably get moved to rustpython-vm proper.

Also the gc module could probably get moved to rustpython-vm proper.

Indeed, in CPython, the gc module is compiled into the interpreter (rather than being dynamically linked):

Python 3.11.0 (main, Oct 25 2022, 13:57:33) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import gc >>> gc.__file__ Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: module 'gc' has no attribute '__file__'. Did you mean: '__name__'?

It would be good for these to have more descriptive names, either a gc_ prefix or a separate module. Also the _gc module could probably get moved to rustpython-vm proper.

how about "gc_bacon" as feature name(as bacon is the papers' author)?

coolreader18 · 2022-11-21T04:28:10Z

derive/src/lib.rs

+/// use on struct with named fields like `struct A{x:i32, y:i32}` to impl `Trace` for datatype
+///
+/// use `#[notrace]` on fields you wish not to trace
+#[proc_macro_attribute]


This could probably be a normal derive() macro, and then you wouldn't have to deal with removing the notrace attr

This could probably be a normal derive() macro, and then you wouldn't have to deal with removing the notrace attr

Yes, #[notrace] is only used four time in th following, still considering whether or not to remove it and hand write Trace:

#[pyclass(module = false, name = "enumerate")] #[derive(Debug)] #[pytrace] pub struct PyEnumerate { #[notrace] counter: PyRwLock<BigInt>, iterator: PyIter, }

#[pyclass(module = false, name = "iterator")] #[derive(Debug)] #[pytrace] pub struct PySequenceIterator { // cached sequence methods #[notrace] seq_methods: &'static PySequenceMethods, internal: PyMutex<PositionIterInternal<PyObjectRef>>, }

#[pyclass(module = false, name = "zip")] #[derive(Debug)] #[pytrace] pub struct PyZip { iterators: Vec<PyIter>, #[notrace] strict: PyAtomic<bool>, }

#[derive(Debug, Clone)] #[pytrace] pub struct ArgMapping { obj: PyObjectRef, #[notrace] methods: &'static PyMappingMethods, }

derive/src/pytrace.rs

vm/src/object/core.rs

coolreader18 · 2022-11-21T04:39:56Z

vm/src/object/core.rs

+    }
+    std::alloc::dealloc(
+        x.cast(),
+        std::alloc::Layout::for_value(x.cast::<PyInner<T>>().as_ref().unwrap()),


Suggested change

std::alloc::Layout::for_value(x.cast::<PyInner<T>>().as_ref().unwrap()),

std::alloc::Layout::for_value(&*x.cast::<PyInner<T>>()),

Maybe also could move this out of the function call and into a variable, just to be safe wrt stacked borrows (not sure if the reference is still "live" while dealloc() runs)

Maybe also could move this out of the function call and into a variable, just to be safe wrt stacked borrows (not sure if the reference is still "live" while dealloc() runs)

Will do, yes I also have doubt on some of my code's soundness, especially with dealloc&drop

coolreader18 · 2022-12-07T02:55:42Z

vm/src/object/core.rs

+                    }
+                )else*
+            };
+        }


Hmm... what if instead of this, PyObjectPayload required Trace (and stuck it in the PyObjVtable), but there's a DummyTrace trait that's safe and marks a object that isn't actually Trace. like:

unsafe trait Trace { const IS_TRACE: bool = true; fn trace(&self, ...); } trait DummyTrace {} // or NoTrace or something unsafe impl<T: DummyTrace> Trace for T { const IS_TRACE: bool = false; fn trace(&self, ...) {} }

And #[pyclass] could by default add a DummyTrace impl, unless specifically asked to pytrace it.

Oh and instead of TraceHelper::is_traceable the vtable has a trace: Option<...> that's None if IS_TRACE is false. Might be faster too, rather than checking 70 different TypeIds it's just one virtual function call.

Hmm... what if instead of this, PyObjectPayload required Trace (and stuck it in the PyObjVtable), but there's a DummyTrace trait that's safe and marks a object that isn't actually Trace. like:

unsafe trait Trace { const IS_TRACE: bool = true; fn trace(&self, ...); } trait DummyTrace {} // or NoTrace or something unsafe impl<T: DummyTrace> Trace for T { const IS_TRACE: bool = false; fn trace(&self, ...) {} }

And #[pyclass] could by default add a DummyTrace impl, unless specifically asked to pytrace it.

Oh and instead of TraceHelper::is_traceable the vtable has a trace: Option<...> that's None if IS_TRACE is false. Might be faster too, rather than checking 70 different TypeIds it's just one virtual function call.

Interesting idea, will try refactor into that

Might rename those to MaybeTrace try_trace and make is_trace a associated const

vm/src/object/core.rs

vm/src/protocol/iter.rs

DimitrisJim · 2023-01-01T19:33:38Z

Maybe we can create a separate CI check that runs the GC with a select few python test files? The idea is to at least have it in and running relevant tests on it while not slowing down the total runtime.

killme2008 · 2023-02-08T07:12:45Z

Is this PR still in review? @youknowone @discord9

youknowone

I am really sorry for late review.
I am not understanding how this gc is working - and I seem to not have enough time to get it before finish review - but I'd like to merge it as an optional feature and a good starting point to enhance gc better in future.
But before merging this change, I hope to minimize changes of CPython-originated code because changed CPython code will be easily forgotten from future developers and it even might be broken when the future enhancement is correct but not compatible with this change.

Lib/test/test_ordered_dict.py

Lib/test/test_sys_setprofile.py

derive-impl/src/pyclass.rs

youknowone · 2023-02-13T08:54:01Z

vm/src/object/core.rs

+        let is_dead = self.is_dead.lock();
+        if *is_dead {
+            return true;
+        }


How does this handled in CPython? Do we really need an additional is_dead field instead of reusing parent field well?

youknowone · 2023-02-13T08:55:46Z

vm/src/object/core.rs

+        }
+        #[cfg(not(feature = "gc_bacon"))]
+        {
+            self.0.ref_count.inc();


we'd better to have unified interface for ref_count handling for #[cfg(not(feature = "gc_bacon"))] in future.

vm/src/object/core.rs

vm/src/object/payload.rs

discord9 · 2023-02-20T03:22:19Z

I am really sorry for late review. I am not understanding how this gc is working - and I seem to not have enough time to get it before finish review - but I'd like to merge it as an optional feature and a good starting point to enhance gc better in future. But before merging this change, I hope to minimize changes of CPython-originated code because changed CPython code will be easily forgotten from future developers and it even might be broken when the future enhancement is correct but not compatible with this change.

I am thinking about update document on how this GC works, perhaps even add it to documents of RustPython?

youknowone · 2023-02-20T05:06:12Z

That will be great in future. I think separating gc feature and reduce effect to non-gc build as it was will be good enough for now.

youknowone · 2023-03-07T19:56:12Z

i rebased it on main branch

discord9 · 2023-03-08T02:37:46Z

i rebased it on main branch

I will fix the CI

youknowone · 2023-03-26T09:50:09Z

vm/src/frame.rs

+            gc_count += 1;
+            if gc_count > 1000 {
+                #[cfg(feature = "gc_bacon")]
+                {
+                    crate::object::try_gc();
+                }
+                gc_count = 0;
+            }


Idk well about gc, so I'd like to learn about the reasoning.
is this the preferred place to put try_gc?
and how the number 1000 is decided? isn't it a potential cause the reason gc is too slow because it is too small?

Idk well about gc, so I'd like to learn about the reasoning. is this the preferred place to put try_gc? and how the number 1000 is decided? isn't it a potential cause the reason gc is too slow because it is too small?

My bad, I should mimic CPython(https://github.com/python/cpython/blob/7703def37e4fa7d25c3d23756de8f527daa4e165/Python/ceval.c#L833), here in handlePending it does necessary check whether to gc or not, I should rewrite this part
According to CPython:

/* Do periodic things, like check for signals and async I/0. * We need to do reasonably frequently, but not too frequently. * All loops should include a check of the eval breaker. * We also check on return from any builtin function. */

Maybe only check for it after certain set of Bytecode is executed or return from a function?

youknowone · 2023-03-27T13:54:23Z

Lib/test/test_ordered_dict.py

+        # TODO: RUSTPYTHON, Need to fix this: somehow after del A, it takes two call to `gc.collect()`
+        # for gc to realize a loop is there and to be collected


Is this still a valid comment for this test?

feat: add double drop check in debug mode fix: use Mutex instead of PyMutex in `ID2TYPE` refactor: cfg cond for `Drop` trait instead feat: add `Trace` trait feat: trace RwLock right&trace tuple feat: `Trace` for `PyDict` feat: `Trace` on `PyIter`&`PyIterReturn`&`PyIterIter` feat: `Trace` on PyEnumerate feat: `Trace` on `ArgCallable` `ArgIterable` `ArgMapping` `ArgSequence` feat: `Trace` on `IterStatus` `PySequenceIterator` `PyCallableIterator` `PositionIterInternal` feat: `Trace` on `PyReverseSequenceIterator` feat: `Trace` on `PyTuple` `PyTupleIterator` `PyTupleTyped` feat: `Trace` on `PyFilter` `PyFunction` `PyBoundMethod` feat: `Trace` on `PyCell` feat: `Trace` on `PyList` `PyListIterator` `PyListReverseIterator` feat: `Trace` on `PyMap` `PyMappingProxy` `MappingProxyInner` feat: `Trace` on PyMemoryViewNewArgs, PyMemoryViewIterator feat: `Trace` on PyProperty, PySet, PySetInner feat: `Trace` on PySlice, PyStaticMethod feat: `Trace` on FuncArgs, KwArgs, PosArgs, OptionalArg feat: `Trace` on PySuper, PySuperNewArgs feat: `Trace` on `PyTraceback` feat: `Trace` for PyBaseException, PyType, PyUnion feat: `Trace` on PyWeakProxy, PyZip, PyBuffer feat: `Trace` on PyMapping, PyNumber, PySequence feat: add `list_traceable` macro fix: right lifetime for `TracerFn` feat: `trace` PyObjectRef&PyRef feat: garbage cycle collector fix: put drop_only in different loop feat: core algorithm of garbage collect feat: add drop_only&dealloc_only to vtable feat: modify core.rs to use gc style: cargo fmt feat: add `try_gc`& gc per frame feat: add `collect` in gc module fix: set black when safe_inc fix: check if is gc-ing in `should_gc` refactor: cfg(gc) for `Drop` trait fix: not add to roots multiple times fix: add judge for if dropped doc: add TODO fix: prevent dealloc cycle garbage early fix: `partially_drop` header later fix: add dealloc guard for deref fix: run `__del__`&drop separately feat: more lock to gc&drop check feat: make gc less freq fix: cfg compile&support attr in partially_drop feat: `pytrace` macro feat: use `#[pytrace]` in some types feat: compact header feat: change gc cond to 10007 obj cnts fix: trace `PyRange` fix: drop ref vtable before dealloc to prevent UB fix: debug check&cfg cond&clippy fix: add ref only after `__del__` is done feat: trace(unsafely ) PyMutex feat: prevent trace PyMutex when not gc feat: change `PyRwlock` back to `PyMutex` fix: testcase test_reference_loop test_unique_composite refactor: early exit of collect_cycles fix: cfg cond feat: gc pause warn msg when wait too long fix: not run __del__ in cycles fix: expected failure for test_unique_composite fix: allow test_ioctl_signed_unsigned_code_param feat: split `drop` to `del`&`weakref` fix: lock cond fix: pause cond so high freq gc not halt all refactor: put impl Collector together feat: drop weak_list later feat: unlock gc pause lock fairly feat: print progress for two long test feat: adjust lock order&not panic fix: let obj handle its weakref's dealloc fix: check stack before pop fix: not leak weakref log: remove some false alarm fix: cfg flag for cond compile fix: cfg flag for no-default-feature fix: use non-block incref test: change gc to 1ms&exit all if wait too long fix: INCREF done right&not gc until last gc done feat: add `gc` feature to root crate doc: TODO for PEP442 del in cycle fix: temporaily add one more `gc.collect()` test: add gc feature in CI refactor: make `mark/scan_roots` associated fn refactor: `free_cycles` fn test: flush progress prompt docs: add TODO for modified testcases refactor: header state's type feat: drop_only log: gc info clippy: drop_only allow unused refactor: rename `gc` feature to `gc_bacon` refactor: remove `allow(unused)` feat: `MaybeTrace` trait feat: add `trace` Meta for `pyclass` proc macro feat: cfg cond flag for `MaybeTrace` feat: add `trace` in vtable fix: add`#[pyclass(trace)` for manual impl trace fix: elide null check in vtable&CR refactor fix: change name in CI tests feat: not use gc in wasm refactor: accord to Code Review doc&fix: explain gc&fix macro fix: test_sys_setprofile

tisonkun · 2024-02-20T08:11:15Z

@discord9 @youknowone do we have some blockers here? Or it's waiting for a rebase and we generally agree on its merge?

discord9 · 2024-02-20T08:28:13Z

@discord9 @youknowone do we have some blockers here? Or it's waiting for a rebase and we generally agree on its merge?

Currently, an old bug stemming from code predating the implementation of garbage collection is causing memory corruption because the garbage collector is indeed collecting that memory. I will open a new pull request once I have fixed those bugs and can reliably pass continuous integration.

youknowone · 2024-04-23T05:14:20Z

@discord9 I am sorry miss the comment. I wish we finally could merge it. If there is any PyTraverse update, could we merge it first? Regardless how GC designed later, it will be useful.

killme2008 · 2024-04-23T06:57:30Z

@discord9 I am sorry miss the comment. I wish we finally could merge it. If there is any PyTraverse update, could we merge it first? Regardless how GC designed later, it will be useful.

+1. It's a first step and it can be improved in future. cc @discord9

discord9 changed the title ~~feat: A simple stop-the-world cycle collector(Draft)~~ feat: A simple stop-the-world cycle collector Sep 22, 2022

discord9 mentioned this pull request Sep 23, 2022

[RFC] GC: Both Stop-the-World and On-the-fly #4158

Open

fanninpm mentioned this pull request Sep 23, 2022

does gc implemented???? #4181

Closed

fanninpm reviewed Sep 23, 2022

View reviewed changes

vm/src/object/gc/trace.rs Outdated Show resolved Hide resolved

discord9 changed the title ~~feat: A simple stop-the-world cycle collector~~ feat: Garbage collect: A simple stop-the-world cycle collector Sep 24, 2022

discord9 force-pushed the main branch from 711e5a2 to 6cf841b Compare September 24, 2022 14:28

discord9 marked this pull request as ready for review September 24, 2022 15:09

discord9 marked this pull request as draft September 25, 2022 14:45

lastmjs mentioned this pull request Oct 3, 2022

Garbage collection demergent-labs/kybra#35

Open

3 tasks

stephanemagnenat mentioned this pull request Oct 11, 2022

[RFC] Support embedding and sandboxing untrusted code #4210

Open

discord9 force-pushed the main branch from 4bc2d5f to 000aa85 Compare October 11, 2022 10:33

This comment was marked as resolved.

Sign in to view

discord9 force-pushed the main branch from befc7d2 to a70720c Compare October 19, 2022 15:14

discord9 force-pushed the main branch from f7cf348 to 3a79613 Compare November 4, 2022 02:47

discord9 changed the title ~~feat: Garbage collect: A simple stop-the-world cycle collector~~ feat: Garbage collect: A simple stop-the-world cycle collector(Redraft) Nov 4, 2022

discord9 changed the title ~~feat: Garbage collect: A simple stop-the-world cycle collector(Redraft)~~ Garbage collect: A simple stop-the-world cycle collector(Redraft) Nov 4, 2022

discord9 changed the title ~~Garbage collect: A simple stop-the-world cycle collector(Redraft)~~ Garbage collect: A simple stop-the-world cycle collector(redrafted) Nov 4, 2022

discord9 marked this pull request as ready for review November 4, 2022 11:57

discord9 changed the title ~~Garbage collect: A simple stop-the-world cycle collector(redrafted)~~ (Redrafted)Garbage collect: A simple stop-the-world cycle collector Nov 5, 2022

coolreader18 reviewed Dec 7, 2022

View reviewed changes

discord9 mentioned this pull request Dec 30, 2022

chore: Update RustPython(With GC) GreptimeTeam/greptimedb#809

Merged

2 tasks

youknowone reviewed Feb 13, 2023

View reviewed changes

youknowone force-pushed the main branch from 45e1bac to 185d167 Compare March 7, 2023 19:55

discord9 force-pushed the main branch from a98b40a to 234d5fe Compare March 23, 2023 06:01

youknowone force-pushed the main branch from c5f8841 to 5f802d2 Compare March 27, 2023 07:16

youknowone reviewed Mar 27, 2023

View reviewed changes

youknowone force-pushed the main branch 3 times, most recently from de08f82 to 0c11636 Compare March 28, 2023 15:36

discord9 and others added 4 commits April 17, 2023 22:03

minimize diff

6e88691

isolate gc from bare object module

823a80f

fix: check buf in drop_slow

9ed5137

discord9 force-pushed the main branch from 0c11636 to 9ed5137 Compare May 5, 2023 07:29

discord9 closed this Feb 20, 2024

tisonkun mentioned this pull request Apr 23, 2024

Python UDF Support GreptimeTeam/greptimedb#3777

Closed

3 tasks

arihant2math mentioned this pull request Jan 28, 2025

Partial Garbage Collection Implementation #5493

Draft

	std::alloc::Layout::for_value(x.cast::<PyInner<T>>().as_ref().unwrap()),
	std::alloc::Layout::for_value(&*x.cast::<PyInner<T>>()),

		# TODO: RUSTPYTHON, Need to fix this: somehow after del A, it takes two call to `gc.collect()`
		# for gc to realize a loop is there and to be collected

(READY FOR REVIEW)Garbage collect: A stop-the-world cycle collector #4180

(READY FOR REVIEW)Garbage collect: A stop-the-world cycle collector #4180

Uh oh!

Conversation

Uh oh!

Summary of GC Algorithm

Progress

future features for GC

GC condition

Basic ideas

Data struct

Example

currently GcTraced types are:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Footnotes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

currently `GcTrace`d types are: