Need help in crawling multiple pages

Hi there!

I want to extract all title articles and dates from this site: https://www.profit.ro/stiri/economie/.

I can crawl one page with no issues...but I don't understand what I am doing wrong when.....trying to go to the next pages...

Here is the code:

import scrapy

class ProfitSpider(scrapy.Spider): name = 'profit' allowed_domains = ['www.profit.ro/stiri/economie/'] start_urls = ['https://www.profit.ro/stiri/economie/']

}

def parse(self, response):
    # Extract article information
    titles = response.xpath('//li//div[@class="row"]//div[@class="col-xs-12 col-sm-8 col-md-9"]//a/@title').extract()
    dates = response.xpath('//li//div[@class="row"]//div[@class="col-xs-12 col-sm-8 col-md-9"]//div[@class="publish-date"]/text()').extract()

    for item in zip(titles, dates,):
        scraped_info = {
            'title': item[0],
            'publish_date': item[1],
        }

        yield scraped_info

    next_page_url = response.xpath("//nav[@class='text-center']//a[@title='pagina urmatoare']/@href").extract_first()
    if next_page_url:
       yield scrapy.Request(response.urljoin(next_page_url), callback=self.parse)