How to build a Fuzzy Inference system with Reddit Data

Initial idea of this project was to collect data relevant to a particular topic in Reddit and build upon a fuzzy inference system to calculate the influence with them to users. This project can be consider as a social media analysis using fuzzy theory and its applications. We use python and relevant libraries for the development.

1. Data Collection with PRAW (Python Reddit API Wrapper)

To gather data from Reddit, we utilized PRAW (Python Reddit API Wrapper) as our primary data gathering tool. PRAW is a powerful library that simplifies interaction with Reddit’s API, enabling us to efficiently access a wide range of data available on the platform. By using PRAW, we were able to scrape extensive datasets, including posts, comments, post score, and user interactions, which are crucial for our analysis. The library’s comprehensive functionality allowed us to filter and collect data based on specific criteria such as keywords, content, and post interactions. This automated data collection process facilitated the systematic extraction of relevant information for our research on the influence of social media on selected topics. Consequently, PRAW proved to be an invaluable tool in obtaining the rich, user-generated content necessary for our fuzzy analysis approach. Below is the usage of this tool.

We need to create a Reddit developer account and a custom app for the following process. Visit https://www.reddit.com/prefs/apps for this procedure. We have to note down Client Id, Client Secret and the name of the app to provide to PRAW.

Details of the Custom App

Let’s install the dependencies we need first :

pip3 install praw
pip3 install json

Following is the usage of PRAW, we include the URL of each post we need to collect data and retrieve the content of the particular post, score, comments and number of comments in this scenario to our analysis. Then the data is stored in JSON format for the ease of use.

import praw
import json

#client_id & client_secret should extract from the developer App of Reddit
reddit = praw.Reddit(client_id='',client_secret='', user_agent='')
url = "" #url of the post
post = reddit.submission(url=url)

result = {"title":post.title,"post":post.selftext,"score":post.score,"
no_of_comments":len(post.comments)}

result["comments"]=[]

for comment in post.comments:
  result["comments"].append(comment.body)

finalResult = json.dumps(result)

2. Building the Fuzzy inference system

Define Variables & Linguistic Variables

After gathering the data, it is time to build the fuzzy system. For this project I have planned to use 3 input variables and 1 output variable for the fuzzy system.

Input variables :

i. Sentiment Score of the content of post

ii.Average Sentiment Score of the comments

iii. Number of Comments

Output variable :

Influence of the post

Then we have to define the range and linguistic variables to the above input/output variables. We can do this by our own intuition, expert knowledge or by a prior research on the topic. Let’s see how we could code this in our code. (Generally sentiment score ranges between [-1, 1] as -1 been fully negative and 1 been fully positive, but for the ease of use I have added 1 to every value to get the range to [0,2] in the below example)

Let’s install the dependencies we need :

pip3 install numpy
pip3 install skfuzzy

import numpy as np
import skfuzzy as fuzz
from skfuzzy import control as ctrl

# Define universe 
sentiment_score_of_post_range = np.arange(0, 2,0.01) #(start,end,step size)
sentiment_score_of_comments_range = np.arange(0, 2,0.01)
no_of_comments_range = np.arange(0,50,1)
influence_range = np.arange(0, 11, 1)

# Define linguistic variables: sentiment score of post
post = ctrl.Antecedent(sentiment_score_of_post_range, 'sentiment_score_of_post')

post['low'] = fuzz.trimf(sentiment_score_of_post_range, [0, 0.5, 1])
post['medium'] = fuzz.trimf(sentiment_score_of_post_range, [0.70, 1.10, 1.50])
post['high'] = fuzz.trimf(sentiment_score_of_post_range, [1.50, 1.70, 2])

# Define linguistic variables: average sentiment score of comments
comments = ctrl.Antecedent(sentiment_score_of_comments_range, 'sentiment_score_of_comments')

comments['low'] = fuzz.trimf(sentiment_score_of_comments_range, [0, 0.3, 0.5])
comments['medium'] = fuzz.trimf(sentiment_score_of_comments_range, [0.5, 0.8, 1.5])
comments['high'] = fuzz.trimf(sentiment_score_of_comments_range, [1.5, 1.7, 2])

# Define linguistic variables: number of comments
no_of_comments = ctrl.Antecedent(no_of_comments_range, 'no_of_comments')

no_of_comments['low'] = fuzz.trimf(no_of_comments_range, [0, 8, 8])
no_of_comments['medium'] = fuzz.trimf(no_of_comments_range, [5, 10, 15])
no_of_comments['high'] = fuzz.trimf(no_of_comments_range, [15, 30, 50])

# Define linguistic variables: influence (output variable)
# note that Antecedent -> Consequent
influence = ctrl.Consequent(influence_range, 'influence', defuzzify_method='centroid')

influence['low'] = fuzz.trimf(influence_range, [0, 0, 3])
influence['medium'] = fuzz.trimf(influence_range, [2, 5, 7])
influence['high'] = fuzz.trimf(influence_range, [7, 10, 10])

Now let’s check our variables and associated linguistic variables by plotting them.

pip3 install matplotlib

I have defined a custom function to plot each graph as they have the same linguistic name to easy demonstration purposes.

def plotMembership(range,variable) : 
    plt.plot(range , variable['low'].mf, label="Low")
    plt.plot(range, variable['medium'].mf,label='Medium')
    plt.plot(range , variable['high'].mf,label="High")

    plt.savefig("1.png") #name of output file

    plt.legend()
    plt.show()

plotMembership(sentiment_score_of_post_range,post)
plotMembership(sentiment_score_of_comments_range,comments)
plotMembership(no_of_comments_range,no_of_comments)
plotMembership(influence_range,influence)

Sentiment Score of the content of post

Average Sentiment Score of the comments

Number of Comments

Influence of the post (Output Variable)

Introduce the Fuzzy rules

Next main phase of the fuzzy inference system is to integrate the fuzzy rules which connects each of the variables. The following rules are used in the project as our intuition and previous user experiments.

rules = []

# Define fuzzy rules
rules.append(ctrl.Rule(post['high'] & comments['high'] & no_of_comments['high'], influence['high']))
rules.append(ctrl.Rule(post['high'] & comments['high'] & no_of_comments['medium'],influence['high']))
rules.append(ctrl.Rule(post['high'] & comments['high'] & no_of_comments['low'],influence['medium']))
rules.append(ctrl.Rule(post['high'] & comments['medium'] & no_of_comments['high'],influence['high']))
rules.append(ctrl.Rule(post['high'] & comments['medium'] & no_of_comments['medium'],influence['high']))
rules.append(ctrl.Rule(post['high'] & comments['medium'] & no_of_comments['low'],influence['medium']))
rules.append(ctrl.Rule(post['high'] & comments['low'] & no_of_comments['high'],influence['high']))
rules.append(ctrl.Rule(post['high'] & comments['low'] & no_of_comments['medium'],influence['high']))
rules.append(ctrl.Rule(post['high'] & comments['low'] & no_of_comments['low'],influence['low']))
rules.append(ctrl.Rule(post['medium'] & comments['high'] & no_of_comments['high'],influence['high']))
rules.append(ctrl.Rule(post['medium'] & comments['high'] & no_of_comments['medium'],influence['high']))
rules.append(ctrl.Rule(post['medium'] & comments['high'] & no_of_comments['low'],influence['high']))
rules.append(ctrl.Rule(post['medium'] & comments['medium'] & no_of_comments['high'],influence['high']))
rules.append(ctrl.Rule(post['medium'] & comments['medium'] & no_of_comments['medium'],influence['low']))
rules.append(ctrl.Rule(post['medium'] & comments['medium'] & no_of_comments['low'],influence['low']))
rules.append(ctrl.Rule(post['medium'] & comments['low'] & no_of_comments['high'],influence['high']))
rules.append(ctrl.Rule(post['medium'] & comments['low'] & no_of_comments['medium'],influence['medium']))
rules.append(ctrl.Rule(post['medium'] & comments['low'] & no_of_comments['low'],influence['low']))
rules.append(ctrl.Rule(post['low'] & comments['high'] & no_of_comments['high'],influence['high']))
rules.append(ctrl.Rule(post['low'] & comments['high'] & no_of_comments['medium'],influence['high']))
rules.append(ctrl.Rule(post['low'] & comments['high'] & no_of_comments['low'],influence['medium']))
rules.append(ctrl.Rule(post['low'] & comments['medium'] & no_of_comments['high'],influence['medium']))
rules.append(ctrl.Rule(post['low'] & comments['medium'] & no_of_comments['medium'],influence['medium']))
rules.append(ctrl.Rule(post['low'] & comments['medium'] & no_of_comments['low'],influence['medium']))
rules.append(ctrl.Rule(post['low'] & comments['low'] & no_of_comments['high'],influence['high']))
rules.append(ctrl.Rule(post['low'] & comments['low'] & no_of_comments['medium'],influence['high']))
rules.append(ctrl.Rule(post['low'] & comments['low'] & no_of_comments['low'],influence['medium']))

# Create fuzzy control system
if(len(rules) > 0):
  influence_ctrl = ctrl.ControlSystem(rules=rules)
  influence_ctrl.view()
  influence_ctrl_sim = ctrl.ControlSystemSimulation(influence_ctrl) 

Defuzzification

Our system is ready now, let’s test it. For this test, I will use random input values to derive the final output value.

# Fuzzification: Provide input values
influence_ctrl_sim.input['sentiment_score_of_post'] = 0.05 
influence_ctrl_sim.input['sentiment_score_of_comments'] = 1.02
influence_ctrl_sim.input['no_of_comments'] = 23

 # Apply fuzzy rules

 influence_ctrl_sim.compute()

Defuzzification is the process of turning our output fuzzy values into crisp values which then can be used for analysis purposes.

# Defuzzification: Obtain crisp output

result = influence_ctrl_sim.output['influence']
print(result)

We can also specify which defuzzification method to use in the skfuzzy library. We have to define it when we add the consequents in the system.(In here when we define the range of ‘Influence’)

influence = ctrl.Consequent(np.arange(0, 11, 1), 'influence', defuzzify_method='centroid') # Add the method accordingly

#centroid = Centroid Method
#bisector = Bisector Method
#mom = Mean of Maximum
#som = Min of Maximum
#lom = Max of Maximum

This is a brief article of how to collect/ scrape data from Reddit as of the time of writing from a particular post and build a fuzzy inference system upon them using python and skfuzzy library.

Hope it helped you in your journey to learn fuzzy theory and systems. :D

Thanks & Good luck !