Scraping and printing titles from Craigslist
up vote
0
down vote
favorite
I've written a very tiny script using class to scrape some titles of products from craigslist. My intention is to make use of __str__() method so that my script can print the result without explicitly calling that very method.
However, when I execute my script in the following way, I get the desired results.
Is this the only way I can print the result or there is anything better to print the result through dunder str method?
import requests
from bs4 import BeautifulSoup
URL = "http://bangalore.craigslist.co.in/search/rea?s=120"
class DataSourcer:
def __init__(self,link):
self.link = link
self.datalist =
def fetch(self):
res = requests.get(self.link)
soup = BeautifulSoup(res.text,"lxml")
for ilink in soup.select("li.result-row a[data-id]"):
self.datalist.append(ilink.text)
# return self.datalist #I do not wish to return "self.datalist" here
def __str__(self):
return self.datalist
if __name__ == '__main__':
crawler = DataSourcer(URL)
crawler.fetch()
print(crawler.__str__()) #I found it working this way
python object-oriented python-3.x web-scraping
edited 15 mins ago
200_success
127k15149412
asked 7 hours ago
MITHU
320110
add a comment |
up vote
0
down vote
favorite
I've written a very tiny script using class to scrape some titles of products from craigslist. My intention is to make use of __str__() method so that my script can print the result without explicitly calling that very method.
However, when I execute my script in the following way, I get the desired results.
Is this the only way I can print the result or there is anything better to print the result through dunder str method?
import requests
from bs4 import BeautifulSoup
URL = "http://bangalore.craigslist.co.in/search/rea?s=120"
class DataSourcer:
def __init__(self,link):
self.link = link
self.datalist =
def fetch(self):
res = requests.get(self.link)
soup = BeautifulSoup(res.text,"lxml")
for ilink in soup.select("li.result-row a[data-id]"):
self.datalist.append(ilink.text)
# return self.datalist #I do not wish to return "self.datalist" here
def __str__(self):
return self.datalist
if __name__ == '__main__':
crawler = DataSourcer(URL)
crawler.fetch()
print(crawler.__str__()) #I found it working this way
python object-oriented python-3.x web-scraping
edited 15 mins ago
200_success
127k15149412
asked 7 hours ago
MITHU
320110
I'm not entirely sure what you're asking, but:print(crawler.__str__())should beprint(str(crawler)).
– Dair
7 hours ago
That is exactly what you have surmised already I'm asking. Your suggested approach threw this errorTypeError: __str__ returned non-string (type list)@Dair . Thanks.
– MITHU
7 hours ago
2
You should instead doreturn str(self.datalist).def __str__should return a string type. Instead, you have returned a list. You can convert the list to a string by callingstr, soreturn str(self.datalist).
– Dair
7 hours ago
Yep, this leads me to a better approach @Dair. Is there any way I can print the result in regular strings other than the list. I know I'm wrong but I tried itfor item in crawler: print(item)and failed miserably. However, I get the result when i try like this (in my existing script)for item in crawler.__str__(): print(item). Thanks.
– MITHU
6 hours ago
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I've written a very tiny script using class to scrape some titles of products from craigslist. My intention is to make use of __str__() method so that my script can print the result without explicitly calling that very method.
However, when I execute my script in the following way, I get the desired results.
Is this the only way I can print the result or there is anything better to print the result through dunder str method?
import requests
from bs4 import BeautifulSoup
URL = "http://bangalore.craigslist.co.in/search/rea?s=120"
class DataSourcer:
def __init__(self,link):
self.link = link
self.datalist =
def fetch(self):
res = requests.get(self.link)
soup = BeautifulSoup(res.text,"lxml")
for ilink in soup.select("li.result-row a[data-id]"):
self.datalist.append(ilink.text)
# return self.datalist #I do not wish to return "self.datalist" here
def __str__(self):
return self.datalist
if __name__ == '__main__':
crawler = DataSourcer(URL)
crawler.fetch()
print(crawler.__str__()) #I found it working this way
python object-oriented python-3.x web-scraping
edited 15 mins ago
200_success
127k15149412
asked 7 hours ago
MITHU
320110
I've written a very tiny script using class to scrape some titles of products from craigslist. My intention is to make use of __str__() method so that my script can print the result without explicitly calling that very method.
However, when I execute my script in the following way, I get the desired results.
Is this the only way I can print the result or there is anything better to print the result through dunder str method?
import requests
from bs4 import BeautifulSoup
URL = "http://bangalore.craigslist.co.in/search/rea?s=120"
class DataSourcer:
def __init__(self,link):
self.link = link
self.datalist =
def fetch(self):
res = requests.get(self.link)
soup = BeautifulSoup(res.text,"lxml")
for ilink in soup.select("li.result-row a[data-id]"):
self.datalist.append(ilink.text)
# return self.datalist #I do not wish to return "self.datalist" here
def __str__(self):
return self.datalist
if __name__ == '__main__':
crawler = DataSourcer(URL)
crawler.fetch()
print(crawler.__str__()) #I found it working this way
python object-oriented python-3.x web-scraping
python object-oriented python-3.x web-scraping
edited 15 mins ago
200_success
127k15149412
asked 7 hours ago
MITHU
320110
edited 15 mins ago
200_success
127k15149412
asked 7 hours ago
MITHU
320110
edited 15 mins ago
200_success
127k15149412
edited 15 mins ago
200_success
127k15149412
edited 15 mins ago
200_success
127k15149412
127k15149412
asked 7 hours ago
MITHU
320110
asked 7 hours ago
MITHU
320110
asked 7 hours ago
MITHU
320110
320110
I'm not entirely sure what you're asking, but:print(crawler.__str__())should beprint(str(crawler)).
– Dair
7 hours ago
That is exactly what you have surmised already I'm asking. Your suggested approach threw this errorTypeError: __str__ returned non-string (type list)@Dair . Thanks.
– MITHU
7 hours ago
2
You should instead doreturn str(self.datalist).def __str__should return a string type. Instead, you have returned a list. You can convert the list to a string by callingstr, soreturn str(self.datalist).
– Dair
7 hours ago
Yep, this leads me to a better approach @Dair. Is there any way I can print the result in regular strings other than the list. I know I'm wrong but I tried itfor item in crawler: print(item)and failed miserably. However, I get the result when i try like this (in my existing script)for item in crawler.__str__(): print(item). Thanks.
– MITHU
6 hours ago
add a comment |
I'm not entirely sure what you're asking, but:print(crawler.__str__())should beprint(str(crawler)).
– Dair
7 hours ago
That is exactly what you have surmised already I'm asking. Your suggested approach threw this errorTypeError: __str__ returned non-string (type list)@Dair . Thanks.
– MITHU
7 hours ago
2
You should instead doreturn str(self.datalist).def __str__should return a string type. Instead, you have returned a list. You can convert the list to a string by callingstr, soreturn str(self.datalist).
– Dair
7 hours ago
Yep, this leads me to a better approach @Dair. Is there any way I can print the result in regular strings other than the list. I know I'm wrong but I tried itfor item in crawler: print(item)and failed miserably. However, I get the result when i try like this (in my existing script)for item in crawler.__str__(): print(item). Thanks.
– MITHU
6 hours ago
I'm not entirely sure what you're asking, but:
print(crawler.__str__()) should be print(str(crawler)).– Dair
7 hours ago
I'm not entirely sure what you're asking, but:
print(crawler.__str__()) should be print(str(crawler)).– Dair
7 hours ago
That is exactly what you have surmised already I'm asking. Your suggested approach threw this error
TypeError: __str__ returned non-string (type list) @Dair . Thanks.– MITHU
7 hours ago
That is exactly what you have surmised already I'm asking. Your suggested approach threw this error
TypeError: __str__ returned non-string (type list) @Dair . Thanks.– MITHU
7 hours ago
2
2
You should instead do
return str(self.datalist). def __str__ should return a string type. Instead, you have returned a list. You can convert the list to a string by calling str, so return str(self.datalist).– Dair
7 hours ago
You should instead do
return str(self.datalist). def __str__ should return a string type. Instead, you have returned a list. You can convert the list to a string by calling str, so return str(self.datalist).– Dair
7 hours ago
Yep, this leads me to a better approach @Dair. Is there any way I can print the result in regular strings other than the list. I know I'm wrong but I tried it
for item in crawler: print(item) and failed miserably. However, I get the result when i try like this (in my existing script) for item in crawler.__str__(): print(item). Thanks.– MITHU
6 hours ago
Yep, this leads me to a better approach @Dair. Is there any way I can print the result in regular strings other than the list. I know I'm wrong but I tried it
for item in crawler: print(item) and failed miserably. However, I get the result when i try like this (in my existing script) for item in crawler.__str__(): print(item). Thanks.– MITHU
6 hours ago
add a comment |
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Code Review Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f209293%2fscraping-and-printing-titles-from-craigslist%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I'm not entirely sure what you're asking, but:
print(crawler.__str__())should beprint(str(crawler)).– Dair
7 hours ago
That is exactly what you have surmised already I'm asking. Your suggested approach threw this error
TypeError: __str__ returned non-string (type list)@Dair . Thanks.– MITHU
7 hours ago
2
You should instead do
return str(self.datalist).def __str__should return a string type. Instead, you have returned a list. You can convert the list to a string by callingstr, soreturn str(self.datalist).– Dair
7 hours ago
Yep, this leads me to a better approach @Dair. Is there any way I can print the result in regular strings other than the list. I know I'm wrong but I tried it
for item in crawler: print(item)and failed miserably. However, I get the result when i try like this (in my existing script)for item in crawler.__str__(): print(item). Thanks.– MITHU
6 hours ago