Lab: Getting started with VSCode, Python and RDFlib: Difference between revisions

From info216
No edit summary
m (Added lab presentation to useful materials)
 
(139 intermediate revisions by 5 users not shown)
Line 1: Line 1:
=Lab 1: Getting started with Pycharm, Python and RDFlib=
==Topics==
# Prepare for programming knowledge graphs with rdflib in Python.
# Get started with basic RDF programming.
 
==Useful materials==
VSCode:
* [https://code.visualstudio.com/docs/python/python-tutorial Getting Started with Python in VS Code]
* [https://code.visualstudio.com/docs/python/environments# Using Python environments in VS Code] (pip is easiest to start with)
 
RDFlib:
* [https://rdflib.readthedocs.io/en/stable/gettingstarted.html Intro to RDFlib]
* [https://rdflib.readthedocs.io/en/stable/intro_to_creating_rdf.html  Intro to creating triples]
* [https://rdflib.readthedocs.io/en/stable/intro_to_parsing.html Serialising and parsing]
* [https://www.youtube.com/watch?v=sCU214rbRZ0 8 min introduction video to RDFlib]


==Topics==
Lab Presentations:
Installation of and introduction to Pycharm and RDFlib.
* [https://docs.google.com/presentation/d/1blXlTTTsL8jqeV5sRLhQuZ-nssNZqH_bjSgjj-kougE/edit?usp=sharing Lab 1 - RDF Presentation]


PyCharm is a Python Integrated Development Environment (IDE) that can be used for writing python code.
RDFlib classes/interfaces and methods:
* Graph (add, close, and perhaps remove methods)
* URIRef
* Literal
* Namespace


RDFLib is a Python library for working with RDF. With this we can create RDF graphs, parse RDF, perform SPARQL queries and more.
==Tasks==
===Install Python, Pip, VSCode, and RDFlib===
'''1)''' You need to have Python version >= 3.7 on your computer. Use the command ''python --version'' in a command/terminal window to check. You can download Python and Pip [https://www.python.org/downloads/ here]. To ensure you have the most recent Pip, you can do
python -m pip install --upgrade pip


==Preparations==
'''2)''' You need to have an Integrated Development Environment (IDE) that supports Python. If you are unsure, you can download the free and open source Visual Studio Code (VSCode) [https://code.visualstudio.com/Download here].
If needed here is an introduction of how to create projects and python files in Pycharm [https://www.jetbrains.com/help/pycharm/creating-and-running-your-first-python-project.html Eclipse IDE - Tutorial by jetbrains.com].


[https://rdflib.readthedocs.io/en/stable/index.html# RDFLib Documentation]- Todays focus are the following:
'''3)''' Create a folder for INFO216. Start VSCode and ''create a workspace'' in the file menu (File Menu --> Save Workspace As) and save it in your folder. Afterwards, on the left side of VSCode, click on the document icon (explorer). Click Open Folder, and open your INFO216 folder. Create a new file with ''.py'' extension.  
* [https://rdflib.readthedocs.io/en/stable/intro_to_creating_rdf.html  Intro to creating triples]
 
* [https://rdflib.readthedocs.io/en/stable/intro_to_parsing.html Loading and saving RDF]
'''4)''' You will be asked to install the Python extension, install it. If you weren't asked, on the left side of VSCode click on the 4 cubes (extension manager). Within here search for Microsoft's Python extension and install it.  


Additional recommended in depth documentation:
'''5)''' If you don't have your terminal open, go to the top menu and click on terminal, and then ''New Terminal''. Check where your terminal window is currently located. The bottom line starting with ''PS'' or ''(base)'' shows where it's located. If you added the folder earlier, then you should be located in your INFO216 folder. However, if the destination after PS is not your INFO216 folder, you need to locate to this folder. You can move through folders with the ''cd'' command in the terminal. For instance, if you are at ''PS C:\Users\YourName>'' and your INFO216 folder is at your desktop, you could type the following ''cd .\Desktop\INFO216\''.    
* [https://rdflib.readthedocs.io/en/stable/rdf_terms.html RDF Terms]
* [https://rdflib.readthedocs.io/en/stable/namespaces_and_bindings.html Namespaces and Bindings]


Useful code [[examples]] codes can be found of this wiki.
'''6)''' If you are correctly located, type in the following command into your terminal window


[[Lecture Notes]]
(Windows)
py -3 -m venv .venv
.venv\scripts\activate


==Relevant RDFlib classes/interfaces (and methods)==
(Mac / Linux)
import [https://rdflib.readthedocs.io/en/stable/py-modindex.html rdflib]:
python3 -m venv .venv
*Graph (add, remove, serialize)
source .venv/bin/activate
*URIRef
*Literal
*NameSpace
*RDF, FOAF


All RDFlib modules can be found [https://rdflib.readthedocs.io/en/stable/py-modindex.html here]
'''7)''' If you get the message ''"... is not digitally signed. You cannot run this script on the current system."'' copy and paste the following in the terminal, and repeat step 6:
I recommend to use browser search (often Ctrl-F) to find the module that you want.)
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope Process


==Tasks==
'''8)''' This should now have created a virtual environment in your folder called ''.venv''. In the bottom right corner you will receive a notification asking your to select the new environment for the workspace folder, select yes. You should now see a ''(.venv)'' in front of PS/(base) in the terminal window. Your virtual environment should now automatically be selected when you open your workspace. However, sometimes you might need to open a new terminal for the (.venv) to appear.
Download and install a Java Runtime Environment (JRE) if you do not have it, for example the most recent update of Java 8 (such as u151): https://www.java.com/en/download/ . Java 9 is also out now, and you can try it out if you want. But because Java 9 is so new, we will stay with Java 8 this spring.


Download and unpack a Java SE Development Kit (JDK) 8 (the most recent update, such as u151/152):
'''9)''' In the terminal, type the following to install RDFlib:
http://www.oracle.com/technetwork/java/javase/downloads/index-jsp-138363.html#javasejdk .
pip install rdflib


'''Here is a smart thing to do from the start:'''  
'''10)''' You might need to close and reopen VSCode for RDFlib to work. You can now ''import rdflib'' into your ''.py'' file, or import specific classes/inferfaces such as ''from rdflib import Namespace, Graph''.
Decide on a folder that you always use for downloading, unpacking and installing your own software. Different software packages you install - like Java, Jena and Eclipse - will go into separate subfolders of this folder. This way you will always know where you have them. On my Windows computer, I put everything under a folder called "C:\Programs\Java". On Linux, I use the folder "/opt/Java". ''It is good practice to use folder names without blank characters (spaces) in them, because they can cause problems for some software packages.''


Download and unpack the Eclipse Oxygen IDE for Java Developers: go to https://eclipse.org/downloads/ and Get Eclipse Oxygen for your computer and operating system. This will download an installer program.  
'''11)''' Right click in your code and click "Run Current File in Interactive Window". When running your program for the first time, you might be asked to install ''ipykernel package'', install it.  


Start the installer program (on Linux, you go to the folder where you installed it and start the "eclipse-inst" program). Choose "Eclipse IDE for Java Developers" and choose an installation folder.
===Programming tasks===
(I use folders called "C:\Programs\eclipse-oxygen" and "/opt/eclipse-oxygen" for this.)
'''Task:'''
Represent the sentences below as triples. Note that some sentences can result in several triples.
* The Mueller Investigation was lead by Robert Mueller.
** It involved Paul Manafort, Rick Gates, George Papadopoulos, Michael Flynn, Michael Cohen, and Roger Stone.
* Paul Manafort was business partner of Rick Gates.
** He was campaign chairman for Donald Trump
** He was charged with money laundering, tax evasion, and foreign lobbying.
** He was convicted for bank and tax fraud.
** He pleaded guilty to conspiracy.
** He was sentenced to prison. 
** He negotiated a plea agreement.
* Rick Gates was charged with money laundering, tax evasion and foreign lobbying.
** He pleaded guilty to conspiracy and lying to FBI.


Start Eclipse from the start menu (on Linux, go to the folder where you installed it and start the "eclipse" program). If you want to, you can make a shortcut to the "eclipse" program from your desktop or task bar.
'''Task:''' Write a program that creates an RDF graph and adds the triples you just created.  


Tell Eclipse that you are going to use the JDK 8 you just downloaded: "Window" -> "Preferences" -> "Java" -> "Installed JREs" -> "Add..."
For the URIs, you can just use an example path like ''http://example.org/''.  
So if you want to represent Donald Trump, the URI could be ''http://example.org/Donald_Trump'', and you can create the resource like this:
from rdflib import URIRef
donaldTrump = URIRef('http://example.org/Donald_Trump')


Download and unpack Jena 3.6.0: https://jena.apache.org/download/ . (I use folders called "C:\Programs\apache-jena" and "/opt/apache-jena" for this.) You can download and unpack the source files too.
You can even use a Namespace so you don't have to write the full URI every time:
from rdflib import Namespace
ex = Namespace('http://example.org/')
donaldTrump = ex.Donald_Trump


While you are at it, you can download and unpack Apache Jena Fuseki too. You will need it later.
'''Task:''' Use the ''serialize'' method of ''rdflib.Graph'' to write out the model in different formats (on screen or to file):
* Turtle (format='ttl')
* N-Triple (format='nt')
* JSON-LD (format='json-ld')
* RDF-XML (format='xml')


Re-start Eclipse and create an Eclipse project for Java (File -> New -> Java Project). You can call it INFO216Lab1 for example. Right click on the new project in the "Package Explorer" and choose "Build Path" -> "Add external Archives..." Now you can copy or link the Jena JAR-files into the project. You find them in: .../apache-jena/lib/*.jar .
Which one is easiest to read? What are the pros and cons of the different formats?
We will look more at some of them later in the course!


Create a Java program that creates a single triple and writes it out. Use the Classes/interfaces and methods listed above. The simplest way to run the program is using the green "play" button in the top Eclipse toolbar.
'''Task:''' Use the simple [https://www.ldf.fi/service/rdf-grapher online RDF grapher] to visualise your model. :isSemantic offers [https://issemantic.net/rdf-visualizer a more advanced RDF visualiser] that you can also test if you want.


Go to the "INFO216 Wiki" page in this wiki and download the file log4j.properties . You can get rid of all the "log4j warnings" by copying the log4j.properties file into exactly the same directory as your Java-files.
'''Task:''' Loop through the triples in the model to print out all triples that have ''pleading guilty'' as predicate. If you have been inconsistent about some predicate or other term, you can first write loops that correct wrong terms everywhere in the model. (''Tip:'' to correct a term in a model, you typically have to first ''remove'' the old triple and then ''add'' a new one.)


==If you have more time...==
==If you have more time...==
If you have more time you can continue extending your graph with some of the stuff from lab 2:
'''Task:''' If you have more time you can continue extending your graph:
''"Cade Tracy lives in 1516 Henry Street, Berkeley, California 94709, USA. He has a B.Sc. in biology from the University of California, Berkeley from 2011. His interests include birds, ecology, the environment, photography and travelling. He has visited Canada and France."''
* Michael Cohen was Donald Trump's attorney.
Try to use as many different methods as possible to create the triples.
** He pleaded guilty for lying to Congress.
* Michael Flynn was adviser to Donald Trump.
** He pleaded guilty for lying to the FBI.
** He negotiated a plea agreement.
 
'''Task:''' According to [https://www.pbs.org/wgbh/frontline/article/the-mueller-investigation-explained-2/ this FRONTLINE article], Gates', Cohen's and Flynn's lying were different and are described in different detail. How can you modify your knowledge graph to account for this?
 
'''Task:''' Write a method (function) that submits your model to https://www.ldf.fi/service/rdf-grapher for rendering and saves the returned image to file.

Latest revision as of 13:18, 19 January 2024

Topics

  1. Prepare for programming knowledge graphs with rdflib in Python.
  2. Get started with basic RDF programming.

Useful materials

VSCode:

RDFlib:

Lab Presentations:

RDFlib classes/interfaces and methods:

  • Graph (add, close, and perhaps remove methods)
  • URIRef
  • Literal
  • Namespace

Tasks

Install Python, Pip, VSCode, and RDFlib

1) You need to have Python version >= 3.7 on your computer. Use the command python --version in a command/terminal window to check. You can download Python and Pip here. To ensure you have the most recent Pip, you can do

python -m pip install --upgrade pip

2) You need to have an Integrated Development Environment (IDE) that supports Python. If you are unsure, you can download the free and open source Visual Studio Code (VSCode) here.

3) Create a folder for INFO216. Start VSCode and create a workspace in the file menu (File Menu --> Save Workspace As) and save it in your folder. Afterwards, on the left side of VSCode, click on the document icon (explorer). Click Open Folder, and open your INFO216 folder. Create a new file with .py extension.

4) You will be asked to install the Python extension, install it. If you weren't asked, on the left side of VSCode click on the 4 cubes (extension manager). Within here search for Microsoft's Python extension and install it.

5) If you don't have your terminal open, go to the top menu and click on terminal, and then New Terminal. Check where your terminal window is currently located. The bottom line starting with PS or (base) shows where it's located. If you added the folder earlier, then you should be located in your INFO216 folder. However, if the destination after PS is not your INFO216 folder, you need to locate to this folder. You can move through folders with the cd command in the terminal. For instance, if you are at PS C:\Users\YourName> and your INFO216 folder is at your desktop, you could type the following cd .\Desktop\INFO216\.

6) If you are correctly located, type in the following command into your terminal window

(Windows)

py -3 -m venv .venv
.venv\scripts\activate

(Mac / Linux)

python3 -m venv .venv
source .venv/bin/activate

7) If you get the message "... is not digitally signed. You cannot run this script on the current system." copy and paste the following in the terminal, and repeat step 6:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope Process

8) This should now have created a virtual environment in your folder called .venv. In the bottom right corner you will receive a notification asking your to select the new environment for the workspace folder, select yes. You should now see a (.venv) in front of PS/(base) in the terminal window. Your virtual environment should now automatically be selected when you open your workspace. However, sometimes you might need to open a new terminal for the (.venv) to appear.

9) In the terminal, type the following to install RDFlib:

pip install rdflib

10) You might need to close and reopen VSCode for RDFlib to work. You can now import rdflib into your .py file, or import specific classes/inferfaces such as from rdflib import Namespace, Graph.

11) Right click in your code and click "Run Current File in Interactive Window". When running your program for the first time, you might be asked to install ipykernel package, install it.

Programming tasks

Task: Represent the sentences below as triples. Note that some sentences can result in several triples.

  • The Mueller Investigation was lead by Robert Mueller.
    • It involved Paul Manafort, Rick Gates, George Papadopoulos, Michael Flynn, Michael Cohen, and Roger Stone.
  • Paul Manafort was business partner of Rick Gates.
    • He was campaign chairman for Donald Trump
    • He was charged with money laundering, tax evasion, and foreign lobbying.
    • He was convicted for bank and tax fraud.
    • He pleaded guilty to conspiracy.
    • He was sentenced to prison.
    • He negotiated a plea agreement.
  • Rick Gates was charged with money laundering, tax evasion and foreign lobbying.
    • He pleaded guilty to conspiracy and lying to FBI.

Task: Write a program that creates an RDF graph and adds the triples you just created.

For the URIs, you can just use an example path like http://example.org/. So if you want to represent Donald Trump, the URI could be http://example.org/Donald_Trump, and you can create the resource like this:

from rdflib import URIRef

donaldTrump = URIRef('http://example.org/Donald_Trump')

You can even use a Namespace so you don't have to write the full URI every time:

from rdflib import Namespace

ex = Namespace('http://example.org/')
donaldTrump = ex.Donald_Trump

Task: Use the serialize method of rdflib.Graph to write out the model in different formats (on screen or to file):

  • Turtle (format='ttl')
  • N-Triple (format='nt')
  • JSON-LD (format='json-ld')
  • RDF-XML (format='xml')

Which one is easiest to read? What are the pros and cons of the different formats? We will look more at some of them later in the course!

Task: Use the simple online RDF grapher to visualise your model. :isSemantic offers a more advanced RDF visualiser that you can also test if you want.

Task: Loop through the triples in the model to print out all triples that have pleading guilty as predicate. If you have been inconsistent about some predicate or other term, you can first write loops that correct wrong terms everywhere in the model. (Tip: to correct a term in a model, you typically have to first remove the old triple and then add a new one.)

If you have more time...

Task: If you have more time you can continue extending your graph:

  • Michael Cohen was Donald Trump's attorney.
    • He pleaded guilty for lying to Congress.
  • Michael Flynn was adviser to Donald Trump.
    • He pleaded guilty for lying to the FBI.
    • He negotiated a plea agreement.

Task: According to this FRONTLINE article, Gates', Cohen's and Flynn's lying were different and are described in different detail. How can you modify your knowledge graph to account for this?

Task: Write a method (function) that submits your model to https://www.ldf.fi/service/rdf-grapher for rendering and saves the returned image to file.