10000 Update API and docs · dahlia/github3.py@5f07230 · GitHub
[go: up one dir, main page]

Skip to content

Commit 5f07230

Browse files
committed
Update API and docs
* Add documentation about GitHubIterator with an example * Add documentation about the new logging introduced, with an example for setting it up * Allow GitHubIterators to be properly refreshed. * Add etag parameter to all of the API iter_* methods
1 parent cb06177 commit 5f07230

File tree

9 files changed

+222
-28
lines changed

9 files changed

+222
-28
lines changed

HISTORY.rst

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,15 @@ History/Changelog
1010
- Add ``iter_commits`` to ``github3.gists.Gist`` as a means of re-requesting
1111
just the history from GitHub and iterating over it.
1212

13-
- Re-organize the library
13+
- Add minimal logging (e.g., ``logging.getLogger('github3')``)
14+
15+
- Re-organize the library.
16+
17+
- Calling ``refresh(True)`` on a ``github3.structs.GitHubIterator`` actually
18+
works as expected now.
19+
20+
- API ``iter_`` methods now accept the ``etag`` argument as the
21+
``GitHub.iter_`` methods do.
1422

1523
- Remove vendored dependency of PySO8601.
1624

docs/examples/iterators.rst

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
.. _iteratorex:
2+
3+
Taking Advantage of GitHubIterator
4+
==================================
5+
6+
Let's say that for some reason you're stalking all of GitHub's users and you
7+
just so happen to be using github3.py to do this. You might write code that
8+
looks like this:
9+
10+
.. code-block:: python
11+
12+
import github3
13+
14+
g = github3.login(USERNAME, PASSWORD)
15+
16+
for u in g.iter_all_users():
17+
add_user_to_database(u)
18+
19+
The problem is that you will then have to reiterate over all of the users each
20+
time you want to get the new users. You have two approaches you can take to
21+
avoid this with :class:`GitHubIterator <github3.structs.GitHubIterator>`.
22+
23+
The First Approach
24+
------------------
25+
26+
You can not call the method directly in the for-loop and keep the iterator as
27+
a separate reference like so:
28+
29+
.. code-block:: python
30+
31+
i = g.iter_all_users():
32+
33+
for u in i:
34+
add_user_to_database(u)
35+
36+
Then after your first pass through your ``GitHubIterator`` object will have an
37+
attribute named ``etag``. After you've added all the currently existing users
38+
you could do the following to retrieve the new users in a timely fashion:
39+
40+
.. code-block:: python
41+
42+
import time
43+
44+
while True:
45+
i.refresh(True)
46+
for u in i:
47+
add_user_to_database(u)
48+
49+
time.sleep(120) # Sleep for 2 minutes
50+
51+
If there are no new users, this won't impact your ratelimit at all. This
52+
mimics the ability to conditionally refresh data on almost all other objects
53+
in github3.py.

docs/examples/logging.rst

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
.. _loggingex:
2+
3+
Using Logging with github3.py
4+
=============================
5+
6+
.. versionadded:: 0.6.0
7+
8+
The following example shows how to set up logging for github3.py. It is off by
9+
default in the library and will not pollute your logs.
10+
11+
.. include:: source/logging_ex.py
12+
:code: python
13+
14+
One thing to note is that if you want more detailed information about what is
15+
happening while the requests are sent, you can do the following:
16+
17+
.. code-block:: python
18+
19+
import logging
20+
urllib3 = logging.getLogger('requests.packages.urllib3')
21+
22+
And configure the logger for urllib3. Unfortunately, requests itself doesn't
23+
provide any logging, so the best you can actually get is by configuring
24+
``urllib3``.
25+
26+
You will see messages about the following in the logs:
27+
28+
* Construction of URLs used in requests, usually in the form:
29+
``('https://api.github.com', 'repos', 'sigmavirus24', 'github3.py')``
30+
* What request is being sent, e.g.,
31+
``POST https://api.github.com/user kwargs={}``
32+
* If JSON is trying to be extracted from the response, what the response's
33+
status code was, what the expected status code was and whether any JSON was
34+
actually returned.

docs/examples/source/logging_ex.py

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
import github3
2+
import logging
3+
4+
# Set up a file to have all the logs written to
5+
file_handler = logging.FileHandler('github_script.log')
6+
7+
# Send the logs to stderr as well
8+
stream_handler = logging.StreamHandler()
9+
10+
# Format the log output and include the log level's name and the time it was
11+
# generated
12+
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
13+
14+
# Use that Formatter on both handlers
15+
file_handler.setFormatter(formatter)
16+
stream_handler.setFormatter(formatter)
17+
18+
# Get the logger used by github3.py internally by referencing its name
19+
# directly
20+
logger = logging.getLogger('github3')
21+
# Add the handlers to it
22+
logger.addHandler(file_handler)
23+
logger.addHandler(stream_handler)
24+
# Set the level which determines what you see
25+
logger.setLevel(logging.DEBUG)
26+
27+
# Make a library call and see the information posted
28+
r = github3.repository('sigmavirus24', 'github3.py')
29+
print('{0} - {0.html_url}'.format(r))

docs/index.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,8 @@ More Examples
4949
examples/git
5050
examples/github
5151
examples/issue
52+
examples/iterators.rst
53+
examples/logging
5254

5355

5456
.. links
@@ -74,6 +76,7 @@ Modules
7476
orgs
7577
pulls
7678
repos
79+
structs
7780
users
7881

7982
Internals

docs/structs.rst

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
.. module:: github3
2+
.. module:: github3.structs
3+
4+
Structures developed for github3.py
5+
===================================
6+
7+
As of right now, there exists only one class in this file, and it is of only
8+
limited importance to users of github3.py. The :class:`GitHubIterator` class
9+
is used to return the results of calls to almost all of the calls to ``iter_``
10+
methods on objects. When conditional refreshing was added to objects, there
11+
was a noticable gap in having conditional calls to those ``iter_`` methods.
12+
GitHub provides the proper headers on those calls, but there was no easy way
13+
to add that to what github3.py returned so it could be used properly. This was
14+
the best compromise - an object the behaves like an iterator regardless but
15+
can also be ``refresh``\ ed to get results since the last request
16+
conditionally.
17+
18+
Objects
19+
-------
20+
21+
.. autoclass:: GitHubIterator
22+
:inherited-members:

github3/api.py

Lines changed: 46 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -74,74 +74,86 @@ def gitignore_templates():
7474
return gh.gitignore_templates()
7575

7676

77-
def iter_all_repos(number=-1):
77+
def iter_all_repos(number=-1, etag=None):
7878
"""Iterate over every repository in the order they were created.
7979
8080
:param int number: (optional), number of repositories to return.
8181
Default: -1, returns all of them
82+
:param str etag: (optional), ETag from a previous request to the same
83+
endpoint
8284
:returns: generator of :class:`Repository <github3.repos.Repository>`
8385
"""
84-
return gh.iter_all_repos(number)
86+
return gh.iter_all_repos(number, etag)
8587

8688

87-
def iter_all_users(number=-1):
89+
def iter_all_users(number=-1, etag=None):
8890
"""Iterate over every user in the order they signed up for GitHub.
8991
9092
:param int number: (optional), number of users to return. Default: -1,
9193
returns all of them
94+
:param str etag: (optional), ETag from a previous request to the same
95+
endpoint
9296
:returns: generator of :class:`User <github3.users.User>`
9397
"""
94-
return gh.iter_all_users(number)
98+
return gh.iter_all_users(number, etag)
9599

96100

97-
def iter_events(number=-1):
101+
def iter_events(number=-1, etag=None):
98102
"""Iterate over public events.
99103
100104
:param int number: (optional), number of events to return. Default: -1
101105
returns all available events
106+
:param str etag: (optional), ETag from a previous request to the same
107+
endpoint
102108
:returns: generator of :class:`Event <github3.events.Event>`\ s
103109
"""
104-
return gh.iter_events(number)
110+
return gh.iter_events(number, etag)
105111

106112

107-
def iter_followers(username, number=-1):
113+
def iter_followers(username, number=-1, etag=None):
108114
"""List the followers of ``username``.
109115
110116
:param str username: (required), login of the person to list the followers
111117
of
112118
:param int number: (optional), number of followers to return, Default: -1,
113119
return all of them
120+
:param str etag: (optional), ETag from a previous request to the same
121+
endpoint
114122
:returns: generator of :class:`User <github3.users.User>`
115123
"""
116-
return gh.iter_followers(username, number) if username else []
124+
return gh.iter_followers(username, number, etag) if username else []
117125

118126

119-
def iter_following(username, number=-1):
127+
def iter_following(username, number=-1, etag=None):
120128
"""List the people ``username`` follows.
121129
122130
:param str username: (required), login of the user
123131
:param int number: (optional), number of users being followed by username
124132
to return. Default: -1, return all of them
133+
:param str etag: (optional), ETag from a previous request to the same
134+
endpoint
125135
:returns: generator of :class:`User <github3.users.User>`
126136
"""
127-
return gh.iter_following(username, number) if username< 10000 /span> else []
137+
return gh.iter_following(username, number, etag) if username else []
128138

129139

130-
def iter_gists(username=None, number=-1):
140+
def iter_gists(username=None, number=-1, etag=None):
131141
"""Get public gists or gists for the provided username.
132142
133143
:param str username: (optional), if provided, get the gists for this user
134144
instead of the authenticated user.
135145
:param int number: (optional), number of gists to return. Default: -1,
136146
return all of them
147+
:param str etag: (optional), ETag from a previous request to the same
148+
endpoint
137149
:returns: generator of :class:`Gist <github3.gists.Gist>`\ s
138150
"""
139-
return gh.iter_gists(username, number)
151+
return gh.iter_gists(username, number, etag)
140152

141153

142154
def iter_repo_issues(owner, repository, milestone=None, state=None,
143155
assignee=None, mentioned=None, labels=None, sort=None,
144-
direction=None, since=None, number=-1):
156+
direction=None, since=None, number=-1, etag=None):
145157
"""List issues on owner/repository. Only owner and repository are
146158
required.
147159
@@ -162,26 +174,32 @@ def iter_repo_issues(owner, repository, milestone=None, state=None,
162174
2012-05-20T23:10:27Z
163175
:param int number: (optional), number of issues to return.
164176
Default: -1 returns all issues
177+
:param str etag: (optional), ETag from a previous request to the same
178+
endpoint
165179
:returns: generator of :class:`Issue <github3.issues.Issue>`\ s
166180
"""
167181
if owner and repository:
168182
return gh.iter_repo_issues(owner, repository, milestone, state,
169183
assignee, mentioned, labels, sort,
170-
direction, since, number)
184+
direction, since, number, etag)
171185
return iter([])
172186

173187

174-
def iter_orgs(username, number=-1):
188+
def iter_orgs(username, number=-1, etag=None):
175189
"""List the organizations associated with ``username``.
176190
177191
:param str username: (required), login of the user
178192
:param int number: (optional), number of orgs to return. Default: -1,
179193
return all of the issues
194+
:param str etag: (optional), ETag from a previous request to the same
195+
endpoint
196+
:returns: generator of
197+
:class:`Organization <github3.orgs.Organization>`\ s
180198
"""
181-
return gh.iter_orgs(username, number) if username else []
199+
return gh.iter_orgs(username, number, etag) if username else []
182200

183201

184-
def iter_repos(login, type='', sort='', direction='', number=-1):
202+
def iter_repos(login, type='', sort='', direction='', number=-1, etag=None):
185203
"""List public repositories for the specified ``login`` or all
186204
repositories for the authenticated user if ``login`` is not
187205
provided.
@@ -198,35 +216,41 @@ def iter_repos(login, type='', sort='', direction='', number=-1):
198216
'desc' otherwise
199217
:param int number: (optional), number of repositories to return.
200218
Default: -1 returns all repositories
219+
:param str etag: (optional), ETag from a previous request to the same
220+
endpoint
201221
:returns: generator of :class:`Repository <github3.repos.Repository>`
202222
objects
203223
"""
204224
if login:
205-
return gh.iter_repos(login, type, sort, direction, number)
225+
return gh.iter_repos(login, type, sort, direction, number, etag)
206226
return iter([])
207227

208228

209-
def iter_starred(username, number=-1):
229+
def iter_starred(username, number=-1, etag=None):
210230
"""Iterate over repositories starred by ``username``.
211231
212232
:param str username: (optional), name of user whose stars you want to see
213233
:param int number: (optional), number of repositories to return.
214234
Default: -1 returns all repositories
235+
:param str etag: (optional), ETag from a previous request to the same
236+
endpoint
215237
:returns: generator of :class:`Repository <github3.repos.Repository>`
216238
"""
217-
return gh.iter_starred(username, number)
239+
return gh.iter_starred(username, number, etag)
218240

219241

220-
def iter_subscriptions(username, number=-1):
242+
def iter_subscriptions(username, number=-1, etag=None):
221243
"""Iterate over repositories subscribed to by ``username``.
222244
223245
:param str username: (optional), name of user whose subscriptions you want
224246
to see
225247
:param int number: (optional), number of repositories to return.
226248
Default: -1 returns all repositories
249+
:param str etag: (optional), ETag from a previous request to the same
250+
endpoint
227251
:returns: generator of :class:`Repository <github3.repos.Repository>`
228252
"""
229-
return gh.iter_subscriptions(username, number)
253+
return gh.iter_subscriptions(username, number, etag)
230254

231255

232256
def create_gist(description, files):

0 commit comments

Comments
 (0)
0