Skip to content

Commit 4169158

Browse files
arunkannawadiarunkannawadi
authored
Merge pull request #369 from scholarly-python-package/develop
Releasing 1.4.5
2 parents 2383b60 + f47e909 commit 4169158

File tree

11 files changed

+275
-20
lines changed

11 files changed

+275
-20
lines changed

‎.github/.codecov.yml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Any changes to this file should be validated before pushing.
2+
# curl --data-binary @.github/.codecov.yml https://codecov.io/validate
3+
codecov:
4+
require_ci_to_pass: false
5+
6+
coverage:
7+
precision: 2
8+
round: down
9+
range: 65...80
10+
# Coverage below 55% is flagged red, and above 80% is green.
11+
12+
status:
13+
project:
14+
default:
15+
branches: null
16+
# Each PR must increase the coverage, but allow a slippage of 5%.
17+
target: auto
18+
threshold: 5%
19+
20+
patch:
21+
default:
22+
branches: null
23+
# Changes have a higher standard and require at least (90-5)% coverage.
24+
target: 90%
25+
threshold: 5%
26+
27+
comment: false

‎.github/CONTRIBUTING.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Developers' Notes
2+
3+
We welcome contributions from you!
4+
5+
If you find a feature in Google Scholar missing in `scholarly`, or find a bug, please feel free to open an appropriate issue based on the two templates you will find.
6+
Additionally, if you are interesting in contributing to the codebase, submit a pull request *after* you open an issue.
7+
8+
## What to contribute
9+
10+
1. If you are new to open-source projects, go through the Issues page and pick one that interests you. The ones with the label "good first issue" are usually simple enough and are new-contributor friendly.
11+
2. If you use `scholarly` in your projects and encounter a bug or need a feature, create an issue first before working on a solution. The issue page can be used for decision-making and design-choices and is meant to utilize your efforts better.
12+
13+
## How to contribute
14+
15+
1. Create a fork of `scholarly-python-package/scholarly` repository.
16+
2. If you add a new feature, try to include tests in already existing test cases, or create a new test case if that is not possible.
17+
3. Make sure the unit tests pass before raising a PR. For all the unit tests to pass, you typically need to setup a premium proxy service such as `ScraperAPI` or `Luminati` (`Bright Data`). If you do not have an account, you may try to use `FreeProxy`. Without a premium proxy, 6 out of 17 test cases typically fail.
18+
4. Submit a PR, with `develop` as your base branch.
19+
5. After an initial code review by the maintainers, the unit tests will be run with the `ScraperAPI` key stored in the Github repository. Passing all tests cases is necessary before merging your PR.

‎.github/ISSUE_TEMPLATE/bug_report.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
---
2+
name: Bug report
3+
about: Create a report to help us improve
4+
title: ''
5+
labels: bug
6+
assignees: ''
7+
8+
---
9+
10+
**Describe the bug**
11+
A clear and concise description of what the bug is.
12+
13+
**To Reproduce**
14+
A minimal code snippet to reproduce the bug. If the bug is due to proxy issues and not exactly reproducible, please provide a code snippet _and_ provide additional information under **Additional context** below.
15+
16+
**Expected behavior**
17+
A clear and concise description of what you expected to happen.
18+
19+
**Screenshots**
20+
If applicable, add screenshots to help explain your problem.
21+
22+
**Desktop (please complete the following information):**
23+
- Proxy service: [e.g. ScraperAPI]
24+
- python version: [e.g. 3.8]
25+
- OS: [e.g. macOS]
26+
- Version [e.g. 1.5]
27+
28+
**Do you plan on contributing?**
29+
Your response below will clarify whether the maintainers can expect you to fix the bug you reported.
30+
- [ ] Yes, I will create a Pull Request with the bugfix.
31+
32+
**Additional context**
33+
Add any other context about the problem here.

‎.github/ISSUE_TEMPLATE/feature_request.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
---
2+
name: Feature request
3+
about: Suggest an idea for this project
4+
title: ''
5+
labels: enhancement
6+
assignees: ''
7+
8+
---
9+
10+
**What feature would you like to request?**
11+
A clear and concise description of what you would like `scholarly` to do.
12+
13+
**Is your feature request related to a problem? Please describe.**
14+
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
15+
16+
**Describe the solution you'd like**
17+
A clear and concise description of what you want to happen.
18+
19+
**Describe alternatives you've considered**
20+
A clear and concise description of any alternative solutions or features you've considered.
21+
22+
**Do you plan on contributing?**
23+
Your response below will clarify if this is something that the maintainers can expect you to work on or not.
24+
- [ ] Yes, I plan to contribute towards this feature in the next couple of days.
25+
26+
**Additional context**
27+
Add any other context or screenshots about the feature request here.

‎.github/PULL_REQUEST_TEMPLATE/pull_request.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
Fixes #Enter the associated issue number.
2+
3+
### Description
4+
Describe your addition briefly to help the reviewers understand your contribution.
5+
6+
## Checklist
7+
8+
- [ ] Check that the base branch is set to `develop` and **not** `main`.
9+
- [ ] Ensure that the documentation will be consistent with the code upon merging.
10+
- [ ] Add a line or a few lines that check the new features added.
11+
- [ ] Ensure that unit tests pass.
12+
If you don't have a premium proxy, ensure that at least 11 tests pass.
13+
The failures should only be due to `MaxTriesExceededException`.

‎.github/workflows/pythonpackage.yml‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ jobs:
2929
run: |
3030
sudo apt-get install -y tor
3131
python3 -m pip install --upgrade pip
32-
pip3 install flake8 pytest
32+
pip3 install flake8 coverage
3333
if [ -f requirements.txt ]; then pip3 install -r requirements.txt; fi
3434
- name: Lint with flake8
3535
run: |
@@ -49,4 +49,15 @@ jobs:
4949
USERNAME: ${{ secrets.USERNAME }}
5050
SCRAPER_API_KEY: ${{ secrets.SCRAPER_API_KEY }}
5151
run: |
52-
python3 -m unittest -v test_module.py
52+
curl --data-binary @.github/.codecov.yml https://codecov.io/validate | head -n 1
53+
coverage run -m unittest -v test_module.py
54+
coverage xml
55+
- name: Upload code coverage
56+
uses: codecov/codecov-action@v2
57+
with:
58+
directory: ./
59+
fail_ci_if_error: true
60+
files: ./coverage.xml
61+
flags: unittests
62+
name: codecov-umbrella
63+
verbose: true

‎CODE_OF_CONDUCT.md‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# Code of Conduct
2+
3+
## Our Pledge
4+
5+
We as members, contributors, and leaders pledge to make participation in our
6+
community a harassment-free experience for everyone, regardless of age, body
7+
size, visible or invisible disability, ethnicity, sex characteristics, gender
8+
identity and expression, level of experience, education, socio-economic status,
9+
nationality, personal appearance, race, religion, or sexual identity
10+
and orientation.
11+
12+
We pledge to act and interact in ways that contribute to an open, welcoming,
13+
diverse, inclusive, and healthy community.
14+
15+
## Our Standards
16+
17+
Examples of behavior that contributes to a positive environment for our
18+
community include:
19+
20+
* Demonstrating empathy and kindness toward other people
21+
* Being respectful of differing opinions, viewpoints, and experiences
22+
* Giving and gracefully accepting constructive feedback
23+
* Accepting responsibility and apologizing to those affected by our mistakes,
24+
and learning from the experience
25+
* Focusing on what is best not just for us as individuals, but for the
26+
overall community
27+
28+
Examples of unacceptable behavior include:
29+
30+
* The use of sexualized language or imagery, and sexual attention or
31+
advances of any kind
32+
* Trolling, insulting or derogatory comments, and personal or political attacks
33+
* Public or private harassment
34+
* Publishing others' private information, such as a physical or email
35+
address, without their explicit permission
36+
* Other conduct which could reasonably be considered inappropriate in a
37+
professional setting
38+
39+
## Enforcement Responsibilities
40+
41+
Community leaders are responsible for clarifying and enforcing our standards of
42+
acceptable behavior and will take appropriate and fair corrective action in
43+
response to any behavior that they deem inappropriate, threatening, offensive,
44+
or harmful.
45+
46+
Community leaders have the right and responsibility to remove, edit, or reject
47+
comments, commits, code, wiki edits, issues, and other contributions that are
48+
not aligned to this Code of Conduct, and will communicate reasons for moderation
49+
decisions when appropriate.
50+
51+
## Scope
52+
53+
This Code of Conduct applies within all community spaces, and also applies when
54+
an individual is officially representing the community in public spaces.
55+
Examples of representing our community include using an official e-mail address,
56+
posting via an official social media account, or acting as an appointed
57+
representative at an online or offline event.
58+
59+
## Enforcement
60+
61+
Instances of abusive, harassing, or otherwise unacceptable behavior may be
62+
reported to the community leaders responsible for enforcement at
63+
Discussions or by emailing any of the contributors listed in setup.py.
64+
All complaints will be reviewed and investigated promptly and fairly.
65+
66+
All community leaders are obligated to respect the privacy and security of the
67+
reporter of any incident.
68+
69+
## Enforcement Guidelines
70+
71+
Community leaders will follow these Community Impact Guidelines in determining
72+
the consequences for any action they deem in violation of this Code of Conduct:
73+
74+
### 1. Correction
75+
76+
**Community Impact**: Use of inappropriate language or other behavior deemed
77+
unprofessional or unwelcome in the community.
78+
79+
**Consequence**: A private, written warning from community leaders, providing
80+
clarity around the nature of the violation and an explanation of why the
81+
behavior was inappropriate. A public apology may be requested.
82+
83+
### 2. Warning
84+
85+
**Community Impact**: A violation through a single incident or series
86+
of actions.
87+
88+
**Consequence**: A warning with consequences for continued behavior. No
89+
interaction with the people involved, including unsolicited interaction with
90+
those enforcing the Code of Conduct, for a specified period of time. This
91+
includes avoiding interactions in community spaces as well as external channels
92+
like social media. Violating these terms may lead to a temporary or
93+
permanent ban.
94+
95+
### 3. Temporary Ban
96+
97+
**Community Impact**: A serious violation of community standards, including
98+
sustained inappropriate behavior.
99+
100+
**Consequence**: A temporary ban from any sort of interaction or public
101+
communication with the community for a specified period of time. No public or
102+
private interaction with the people involved, including unsolicited interaction
103+
with those enforcing the Code of Conduct, is allowed during this period.
104+
Violating these terms may lead to a permanent ban.
105+
106+
### 4. Permanent Ban
107+
108+
**Community Impact**: Demonstrating a pattern of violation of community
109+
standards, including sustained inappropriate behavior, harassment of an
110+
individual, or aggression toward or disparagement of classes of individuals.
111+
112+
**Consequence**: A permanent ban from any sort of public interaction within
113+
the community.
114+
115+
## Attribution
116+
117+
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
118+
version 2.0, available at
119+
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
120+
121+
Community Impact Guidelines were inspired by [Mozilla's code of conduct
122+
enforcement ladder](https://github.com/mozilla/diversity).
123+
124+
[homepage]: https://www.contributor-covenant.org
125+
126+
For answers to common questions about this code of conduct, see the FAQ at
127+
https://www.contributor-covenant.org/faq. Translations are available at
128+
https://www.contributor-covenant.org/translations.

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
1-
[![Python package](https://github.com/scholarly-python-package/scholarly/workflows/Python%20package/badge.svg?branch=master)](https://github.com/scholarly-python-package/scholarly/actions?query=branch%3Amaster)
2-
1+
[![Python package](https://github.com/scholarly-python-package/scholarly/workflows/Python%20package/badge.svg?branch=main)](https://github.com/scholarly-python-package/scholarly/actions?query=branch%3Amain)
2+
[![codecov](https://codecov.io/gh/scholarly-python-package/scholarly/branch/main/graph/badge.svg?token=0svtI9yVSQ)](https://codecov.io/gh/scholarly-python-package/scholarly)
33
[![Documentation Status](https://readthedocs.org/projects/scholarly/badge/?version=latest)](https://scholarly.readthedocs.io/en/latest/?badge=latest)
44

55
# scholarly

‎scholarly/publication_parser.py‎

Lines changed: 11 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ def _get_total_results(self):
6565
return None
6666

6767
for x in self._soup.find_all('div', class_='gs_ab_mdw'):
68-
# Accounting for different thousands separators:
68+
# Accounting for different thousands separators:
6969
# comma, dot, space, apostrophe
7070
match = re.match(pattern=r'(^|\s*About)\s*([0-9,\.\s’]+)', string=x.text)
7171
if match:
@@ -126,7 +126,7 @@ def _citation_pub(self, __data, publication: Publication):
126126
and not year.text.isspace()
127127
and len(year.text) > 0):
128128
publication['bib']['pub_year'] = year.text.strip()
129-
129+
130130
return publication
131131

132132
def get_publication(self, __data, pubtype: PublicationSource)->Publication:
@@ -169,7 +169,7 @@ def _get_author_id_list(self, authorinfo_inner_html):
169169
match = re.search('\\?user=(.*?)&', author_html)
170170
if match:
171171
author_id_list.append(match.groups()[0])
172-
else:
172+
else:
173173
author_id_list.append("")
174174
return author_id_list
175175

@@ -235,22 +235,19 @@ def _scholar_pub(self, __data, publication: Publication):
235235
if publication['bib']['abstract'][0:8].lower() == 'abstract':
236236
publication['bib']['abstract'] = publication['bib']['abstract'][9:].strip()
237237

238+
publication['url_scholarbib'] = _BIBCITE.format(cid, pos)
239+
sclib = self.nav.publib.format(id=cid)
240+
publication['url_add_sclib'] = sclib
241+
238242
lowerlinks = databox.find('div', class_='gs_fl').find_all('a')
239243

240244
publication["num_citations"] = 0
241245

242246
for link in lowerlinks:
243-
if (link is not None and
244-
link.get('title') is not None and
245-
'Cite' == link.get('title')):
246-
publication['url_scholarbib'] = _BIBCITE.format(cid, pos)
247-
sclib = self.nav.publib.format(id=cid)
248-
publication['url_add_sclib'] = sclib
249-
250247
if 'Cited by' in link.text:
251248
publication['num_citations'] = int(re.findall(r'\d+', link.text)[0].strip())
252249
publication['citedby_url'] = link['href']
253-
250+
254251
if 'Related articles' in link.text:
255252
publication['url_related_articles'] = link['href']
256253

@@ -262,7 +259,7 @@ def _scholar_pub(self, __data, publication: Publication):
262259

263260
def fill(self, publication: Publication)->Publication:
264261
"""Populate the Publication with information from its profile
265-
262+
266263
:param publication: Scholar or Citation publication container object that is not filled
267264
:type publication: PublicationCitation or PublicationScholar
268265
"""
@@ -310,15 +307,15 @@ def fill(self, publication: Publication)->Publication:
310307
# try to find all the gsh_csp if they exist
311308
abstract = val.find_all(class_='gsh_csp')
312309
result = ""
313-
310+
314311
# append all gsh_csp together as there can be multiple in certain scenarios
315312
for item in abstract:
316313
if item.text[0:8].lower() == 'abstract':
317314
result += item.text[9:].strip()
318315
else:
319316
result += item.text
320317

321-
if len(abstract) == 0: # if no gsh_csp were found
318+
if len(abstract) == 0: # if no gsh_csp were found
322319
abstract = val.find(class_='gsh_small')
323320
if abstract:
324321
if abstract.text[0:8].lower() == 'abstract':

‎setup.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55

66
setuptools.setup(
77
name='scholarly',
8-
version='1.4.4',
8+
version='1.4.5',
99
author='Steven A. Cholewiak, Panos Ipeirotis, Victor Silva, Arun Kannawadi',
1010
author_email='steven@cholewiak.com, panos@stern.nyu.edu, vsilva@ualberta.ca, arunkannawadi@astro.princeton.edu',
1111
description='Simple access to Google Scholar authors and citations',

0 commit comments

Comments
 (0)